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Abstract 

We consider the average-consensus problem in a multi-node network of finite size. Communication between 
nodes is modeled by a sequence of directed signals with arbitrary communication delays. Four distributed algorithms 
that achieve average-consensus are proposed. Necessary and sufficient communication conditions are given for each 
algorithm to achieve average-consensus. Resource costs for each algorithm are derived based on the number of scalar 
values that are required for communication and storage at each node. Numerical examples are provided to illustrate 
the empirical convergence rate of the four algorithms in comparison with a well-known "gossip" algorithm as well 
as a randomized information spreading algorithm when assuming a fully connected random graph with instantaneous 
communication. 



Glossary 
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BM, Bench-Mark Algorithm, proposed here in (|T0j) - (fl4|) . 

DA, Distributed- Averaging Algorithm, proposed here in (TO]) — |19|. 

OH, One-Hop Algorithm, proposed here in (f20| - (|24|) . 

DDA, Discretized Distributed- Averaging, proposed here in ([27]) — (|32l) . 

Gossip, Gossip Algorithm, proposed in [3|, also defined in ()156l) — (|158p . 

RIS, Randomized Information Spreading, proposed in |fT9l . 

ARIS, Adapted Randomized Information Spreading, defined in (|160l) — (|166p . 

SVSC, a "singly V-strongly connected" communication sequence, defined in ((34|) . 

SVCC, a "singly V-completely connected" communication sequence, defined in (|36) 

IVSC, an "infinitely V-strongly connected" communication sequence, defined in (j35 
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• IVCC, an "infinitely V-completely connected" communication sequence, defined in ((37]) . 



I. Introduction 

Average-consensus formation implies that a group of distinct nodes come to agree on the average of their initial 
values, see 0, Q, iflOl . fl9l . ED, (29], |6| for related work. Past results indicate that obtaining average-consensus 
on a set of of n initial vectors each with dimension d requires at least one of the following assumptions: 

• storage and communication of a set with cardinality upper bounded by 0(nd) at each node, e.g. the "flooding" 
method described for example in (4), |16| (where O denotes the Landau big Oh) 

• construction of directed acyclical graphs in the network topology (6), Q], lfl6l 

« knowledge at the transmitting node of the receiving node's identity H171 . (6), JO, lfl6l 

• instantaneous communication as well as knowledge at the transmitting node of its out-degree lfl2l . (9l 

• strictly bi-directional and instantaneous communication (5), (8), (7), IflOl . ifTTl . (291 

• instantaneous communication and symmetric probabilities of node communication [ 10"], ifTZl 

• an approximate average-consensus based on randomized number generation fl9l . O, Q 

• pre-determined bounds on the communication delays as well as the use of averaging-weights that are globally 
balanced and pre-determined off-line (28l. lfT5l. Ifl4l. |[T3l. IBl. ET\. 

This paper proposes four algorithms to solve the average-consensus problem under weaker communication 
conditions than those listed above. We denote the four algorithms Bench-Mark (BM), Distributed- Averaging (DA), 
One-Hop (OH), and Discretized Distributed- Averaging (DDA) . 

(i) The BM algorithm is based on the "flooding" method described in 10), (TBI . We show in Theorems I4.2[|4.3l that 
the BM algorithm achieves average-consensus given the weakest communication condition necessary for average- 
consensus under any distributed algorithm (this is the SVSC condition, defined in Sec.IV). 

(ii) As the main result, we show in Theorem 14.51 that the DA algorithm (a reduction of the BM algorithm) can 
achieve average-consensus under a recurring SVSC condition (this is the IVSC condition, defined in Sec.IV). By 
"recurring" we mean that for an infinite set of disjoint time intervals, the SVSC condition occurs on each interval. 
Previous results based on iterative averaging (e.g. J3), IflOl . (26], (2TJ, (25)) require special cases of the IVSC 
condition. 

(iii) The OH and DDA algorithms can be viewed respectively as simplified versions of the BM and DA algorithms. 
We will show that analogous results hold under these algorithms with respect to the SVCC and IVCC conditions 
defined in Sec.IV. 

In contrast to earlier work, the main results of this paper show that under general uni-directional connectivity 
conditions on the communication sequence, each proposed algorithm achieves average-consensus in the presence 
of 

• arbitrary communication delays, 
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• arbitrary link failures. 

Another distinct contrast each of the proposed algorithms has with previously considered consensus algorithms is, 
> each node will know exactly when the true average-consensus estimate has been locally obtained, regardless 
of the communication pattern between nodes. 
Of course our general results come at a price. The main drawback of three of our proposed algorithms (DA, DDA, 
OH) is that they require local storage and transmission upper bounded by 0(n + d), where we recall that n is 
the network size and d is the dimension of the initial consensus vectors. Most average-consensus algorithms in the 
literature require only 0{d) costs, however they also require assumptions such as instantaneous communication, 
pre-determined averaging weights, or control at the transmitting node of the receiver identity. Our algorithms (DA, 
DDA, OH) require none of these assumptions and are particularly advantageous for average-consensus involving 
d > n distinct scalars. In this case the transmission and storage cost of each algorithm is O(d), hence the assumption 
of relatively weak communication conditions can be leveraged against past algorithms. There are several examples 
in the literature where d > n. For instance, if each node observes a ^pn dimensional process or parameter in noise, 
then a distributed Kalman filter or maximum-likelihood estimate requires an average-consensus on d = n + y/n 
scalars (see |20|, 0). 

The first algorithm we consider (BM) is an obvious solution and is presented here only because (i) it serves as a 
bench-mark for the other algorithms (ii) the communication conditions necessary for its convergence will be used 
in our main results, and (iii) its formal description has many of the same properties as the other three proposed 
algorithms. The BM algorithm requires local storage and transmission of 0(nd) and has optimal convergence rate 
(see Theorem I42P31 in Sec.IV). 

A. Relation to Past Work 

Since the literature in consensus formation is substantial, we now give an overview of existing results and 
compare them with the results of this paper. For sufficiently small e > 0, if Xk = [x\, x\, . . . , x r k L ] € l n denotes the 
network state at time k, then [21 1 proves that each element in the sequence {xt = (I — eCk)xk-i , k = 1, 2, . . .} 
converges asymptotically to Y17=i x o/ n ^ eacn 8 ra P n Laplacian Ck in the sequence {Ck € R™ x " : k = 1, 2, . . .} 
is balanced and induces a strongly connected graph. The work |26| generalizes this result by allowing e to decrease 
at a sufficiently slow rate, and assuming only that there exists some integer j3 such that the union of all graph 
Laplacians over the interval [k/3, (k + — 1] induces a strongly connected graph for each k > 1. However, 
each graph Laplacian in ll26ll is still assumed to be balanced, and as is typical of many interesting papers on 
average-consensus, neither ETI nor ||26| explain any method by which the nodes can distributively ensure each Ck 
is balanced while the sequence {Ck ■ k = 1, 2, . . .} is occurring. Hence the results of these works assume that all 
averaging weights are globally pre-determined off-line, or in other words, every node in the network is assumed to 
know what averaging weights they should locally be using at each iteration to guarantee the resulting Ck is globally 
balanced. 
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In contrast, J5j proposes a "Metropolis" algorithm that requires a "two-round" signaling process wherein nodes 
can distributively compute local averaging weights. It is shown in [5 | that each element in the sequence {x^ = 
(I — e£k)xk-i , k = 1, 2, . . .} converges asymptotically to 53i=i x h/ n under mild connectivity conditions on the 
sequence {Ck ■ k = 1,2, ...}. The work (9] also proposes a distributed algorithm that does not require pre- 
determined averaging weights, however [9 j assumes each transmitting node knows the number of nodes that will 
receive its message, even before the message is transmitted. Similarly, the algorithm in assumes bi-directional 
communication, and furthermore each of the stated results in GD . ||26l , 151 , (9) assume that the communication is 
instantaneous. In contrast, only |2~T1 . ||9l and (5J assume the communication is noiseless. 

The results in this paper do not assume instantaneous or bi-directional communication, nor do they assume that 
the transmitting node knows what node will receive their message or when. However, our results do require noiseless 
communication. The issue of noisy communication can be treated as a "meta-problem" that may be super-imposed 
upon the framework considered here. Similar to our current approach, there has been much research that assumes 
noiseless communication (for instance lfl2l . ifTTl . |fl9l ). Conversely, there is a growing body of work that considers 
average-consensus formation in specifically noisy communication settings IfTTII . [26|, [14|. 

Works such as 0, ED, (27), (B), (24), 10, ASH, ED, ED, G3) require node communication properties that 
are special cases of the IVSC condition defined in Sec lIV-Al However, besides the flooding BM method and the 
DA algorithm proven to converge here, the only other known algorithm that can even be conjectured to almost 
surely obtain average-consensus under all IVSC sequences is a specific adaptation of the randomized information 
spreading (RIS) algorithm proposed in [19]. Our adaptation of this algorithm is referred to as ARIS and is detailed 
in Sec.VII-B of Appendix VII. We note, however, that the lower bound on convergence rate derived in |fl9l assumes 
instantaneous bi-directional communication, and furthermore, any version of the RIS algorithm assumes that the 
initial consensus variables are all positive valued. The Gossip algorithm proposed in Q as well as the ARIS 
algorithm are used as points of reference for the four algorithms proposed in this paper. In Sec IV- Al we compare the 
resource costs of all six algorithms, and in See l V-BI the performance of each algorithm is illustrated by simulation 
under various randomized communication sequences when assuming a full network graph. 

We note that lfl8l . 11271 . (24l . (2), ll23l do not guarantee the final consensus value will equal the initial average; 
O proposes an update rule that assumes instantaneous bi-directional communication; and |25l . fT51 . ll2"Tl assume 
pre-determined bounds on the communication delays as well as averaging-weights that are globally balanced at 
each iteration (see Theorem 5, J25)). In contrast, Theorem l4.5l in Sec. IV states that the DA algorithm will guarantee 
average-consensus under all IVSC sequences, regardless of the communication delays and without requiring any 
pre-determined balancing of the averaging weights. We note that a distinct feature that all four of the proposed 
algorithms have is that they each require the local initial consensus vector to be stored in the respective database 
of each node until average-consensus is locally obtained at that node. Without this property, each algorithm could 
still ensure a "consensus formation" under the exact same communication conditions assumed in our main results, 
however the final consensus value would not necessarily equal the initial average, which is desirable in most 
applications [3|, [4|, (6), Q], lfl6l . EOl . In further contrast to past results, the proofs of convergence used in this 
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work do not rely on matrix theory J5), ||2TI . Lyapunov techniques [18], [28 1, or stochastic stability |10|, |25|, Q~3); 
instead, for our main results we obtain a variety of lower bounds on the "error" reduction and show that under 
(deterministic) recurring connectivity conditions an average-consensus will asymptotically obtain in the L? norm. 
As a final note, we clarify that the communication is assumed to be causal, hence a signal cannot be received 
at node i before it has left node j. Given this assumption, our framework considers every possible sequence of 
signals, hence any realization of a (causal) stochastic communication model is a special case of our deterministic 
framework. 

B. Outline 

Sec. II formulates the problem statement as well as our assumptions regarding the node communication and 
algorithm framework. Sec.III defines the four proposed algorithms. Sec. IV states the communication conditions 
that are necessary and sufficient for each algorithm to obtain average-consensus. Sec.V considers the numerical 
implementation of the four algorithms together with the two comparison algorithms Gossip and ARIS. Resource 
costs are given in Sec lV-AI and numerical simulations are presented in Sec lV-BI A summary of the results and 
suggested future work is provided Sec. VI. The Appendix VII presents the proofs of all theorems, the two comparison 
algorithms, four conjectures, the resource cost derivations, and an important example. 

II. The Average-Consensus Problem and Algorithm Assumptions 

This section formulates the average-consensus problem and lists the assumptions regarding communication 
between nodes. Sec lII-Al below defines the graph theoretic model for consensus formation that will be subsequently 
analyzed, and Sec lII-BI defines the class of the distributed algorithms we consider. Sec lII-CI details the remaining 
assumptions that will be made on the node communication and computational abilities, and also explains the 
technique by which the four proposed algorithms will obtain average-consensus. 

A. Problem Formulation 

Let t > denote time (the results of this paper assume a continuous time framework; any discrete time index 
is a special case of this framework). At initial time t — 0, consider a finite set of arbitrarily numbered nodes 
V = {1, . . . ,n} and a set of d-dimensional vectors {sj(0) G K d : i G V}. The set {sj(0) G R d : i G V} is 
referred to as the set of "initial consensus vectors". Suppose each node can locally store a "knowledge set" /Cj(i) 
that consists of a group of scalars each with a distinct meaning. 

• (Al): (knowledge set assumption) At any time t > 0, each node i G V is equipped with a device that can 
update and store a "knowledge set" JCi(t). For each i 6 V, the knowledge set JCi(t) may have a time-varying 
cardinality. 

Next we assume that a set S % * (t$ , t( ) can be transmitted from node j at a time denoted t$ > 0, and received at 
node i at a time denoted as tff , where due to causality t( > . We refer to S'^ (t 1 ^ , t± ) as a "signal", or "signal 
set". 
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• (A2): (signal set assumption) At any time > 0, each node j E V has the ability to transmit a "signal set" 
S 1J ,t % i) C K.j(t l Q) that will be received at some node i E V at time t l ( > t^. 

As our final condition, we assume that at t = each node i E V will "know" its unique node identifier value i, 
the network size n, and the respective initial consensus vector Sj(0). This is formalized as, 

• (A3): At time t = 0, the knowledge set /C,(0) of each node i € V satisfies /Ci(0) D {i, n, Sj(0)}. 
Definition 2.1: Under (A1)-(A3), the average-consensus problem is solved at some time instant t if and only if 

(iff) the average of the initial consensus vectors, 

1 " 

5 (°) = -I>( ) > c 1 ) 

is contained in the knowledge set JCi(t) of all nodes i e V. We say that a specific node i has obtained average- 
consensus at time t iff s(0) € JCi(t). □ 
The four algorithms analyzed in this paper can be adapted so that the size of the network n and unique node 
identifier i e V can be computed during the averaging process, and thus they need not be initially known at any 
node. For simplicity, however, it will be assumed that the network size n and unique node identifier i are initially 
known at each node i E V. 

B. The Distributed Algorithm and Local Consensus Estimates 

To provide a unifying theme for the algorithms discussed in this paper, we first define the class of distributed 
algorithms that will be considered for consensus formation. Given the assumptions (Al) — (A2), we define a 
"distributed algorithm" in terms of its "knowledge set updating rule" /*:{•} together with a "signal specification 
rule" fs{-}- The knowledge set updating rule /jc{ - } defines the effect that a set of signals has on the knowledge 
set Kiiff) at the receiving node i, whereas the signal specification rule fs{-} defines the elements contained in 
the signal £* J (t^ , t % l ) given as a function of the knowledge set K,j(t 1 ^) at the transmitting node j. If we assume 
that upon reception of a set of signals At > time elapses before the knowledge set is fully updated, then the 
class of distributed algorithms we consider may be defined as follows. 



Class of Distributed Algorithms under (A1)-(A2): 

Knowledge Set Updating Rule: fc : JC t (t[ J ) [j S l3 {ti j ,t[ J ) i — > Kitfj+M) (2) 

*o 3 <*i J ■ iev 

Signal Specification Rule: f s : fCjifj) \ — > S ij (tg (3) 



The algorithms in this paper will assume that every knowledge set JCi (t) contains a local "consensus estimate" 
Si(t) E K d that represents the "belief" of node i in regard to the average-consensus s(0) defined in ([TJ), 

Ki(t) D {si{t)} , V i E V , V t > . (4) 
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Notice that using a local consensus estimate Sj(t) is necessary for any algorithm seeking to solve the average- 
consensus problem; if there is no local consensus estimate then there is no means by which the knowledge set of 
any node can contain s(0) and hence the problem stated in Definition 12. II is ill-posed. 

In contrast from any known past consensus algorithm, the four proposed algorithms here will also assume that 
at any time t > the knowledge set JCi(t) of each node i E V will contain a local "normal consensus estimate" 
Vi(t) e R n , 

£i(t) 2 {«<(*)} , V«€ V , Vt >0 . (5) 

Combining (A3), Q, and Q we then have the initial knowledge set for each of the four proposed algorithms, 

. (A4): )C i (0) = {i,n,s i (0),s i (0),v i (0)} , VieV. 
Define the "error" of the consensus estimate s,(t) as follows, 

^(0)(3i(*)) = lV(S<(*)-*(0)) a | , 

where s(0) is defined in (UJ. Denote the "network consensus error" 2~27=i -^s(o) • We conclude this section 

with the following definition of what constitutes the solution to the average-consensus problem. 

Definition 2.2: The average-consensus problem is solved at time t iff the network consensus error is zero, that 

is, 

n 

2 s *w (*<(*)) =0 • (6) 

□ 

C. Node Communication and Update Assumptions 

Below we detail the node communication and update assumptions that will be used by each of the four proposed 
algorithms. The final update condition we propose will explain the technique by which all four algorithms achieve 
average-consensus. 

For any t > 0, define f (+) as the right-hand limit of t, that is t(+) = lim T _ > . t + T. To construct a suitable 
average-consensus algorithm we assume the following conditions on the node communication and knowledge set 
updates: 

• (A5): at no time does any node j G V have the ability to know when the signal S 1 ^ (iff , t j 3 ) is transmitted, 
when the signal S* iJ (t^ ,t^) is received, or what node i € V will receive the signal S v (t l Q , £f). 

• (A6): at no time does any node i e V have the ability to control when the signal S 1 ^ (t^ , t l ( ') is received, what 
node jeV transmitted the signal S 1 ^ (t^ , t( ) , or when the signal S v (i^ , t( ) was transmitted. 

> (A7): each knowledge set satisfies /Cj(t(+)) = /C^ (t) at any time t > that node i does not receive a signal 

(recall that denotes the right-hand limit of t). 
m (A8): when a signal S 11 (t % Q is received, the knowledge set fCi(£f) of the receiving node is updated in an 

arbitrarily small amount of time. 

• (A9): at most one signal can be received and processed by a given node at any given time instant. 

DRAFT 



8 



Note that (A5)-(A6) imply the communication process is a priori unknown at every node. The assumption (A7) 
implies that the node knowledge set /Cj(t) can only change if a signal is received at node i, and (A8) implies that 
any signal (tg , t( ) transmitted from node j is allowed to contain information that has been updated by a signal 
received at node j at any time preceding . Notice that (A8) is realistic since all four of the proposed algorithms 
will require only a few arithmetic operations in the update process. Assumption (A9) is a technical requirement 
that simplifies the proofs of convergence. Together (A8) and (A9) imply the knowledge set updating rule /jc{ - } 
defined in (|2|) reduces to, 

f K : /C,(t«)U5«(t« t«) — > W(+)). (7) 

Each algorithm we propose can be easily adapted if (A8)-(A9) were relaxed, however this would be at the expense 
of simplicity in both our framework and analysis. 

As our final update condition, we require that the local consensus estimate Si(t^) at the receiving node i, as 
defined above Q, is updated based on the updated normal consensus estimate u»(ti J (+)) via the relation, 

*(#'(+)) = SSi (#'(+)) , (8) 

where S = [s x (0); s 2 (0); • • • ; s„(0)] G R dxn . Under © it is clear that s z (t\ 3 (+)) = s(0) if v t (t\ 3 (+)) = ±1„, 
where 1„ G denotes a vector that consists only of unit-valued elements. Thus, in terms of Definition 12.21 under 
(HJ) and © the average-consensus problem is solved at time t if Vi(t) = — 1„ for all nodes i 6 V. Motivated by 
this fact, we propose for each of the four algorithms that the normal consensus estimate t), (t) is updated based on 
the following optimization problem, 

«i(*l'(+)) = a rg£ min (u-±l„) 2 , 

s.t. © holds, given JC, (t\ j ) (J , ^ J ) . 

Note that from Q and (|5]) each node i € V will know it has obtained the true average-consensus value Sj (t) = s(0) 
when the local normal consensus estimate Vi(t) satisfies the condition Vi(t) = In the next section we will 

define the knowledge set updating rule /jc{ - } and signal specification rule fs{-} for each of the four algorithms. We 
shall find that for each algorithm the update problem (|9]) reduces to a least-squares optimization and a closed-form 
expression can be obtained for updated normal consensus estimate (£*/(+)). 

III. Distributed Average-Consensus Algorithms 

With the above definitions, we are now ready to describe the four distributed algorithms that achieve average- 
consensus. The details of the two comparison algorithms can be found in Sec.VII-B of Appendix VII. All six 
algorithms (the four presented below and the two comparison algorithms in Sec.VII-B) are defined using the 
abstract definition of the "distributed algorithm" given in Sec. II, that is j3]), @- This section sets the stage for the 
convergence theorems presented in Sec. IV. 
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A. Algorithm 1: Bench-Mark (BM) 

The BM algorithm obtains average-consensus trivially; we propose it formally because the remaining three 
algorithms are specific reductions of it, and also because it requires communication conditions that will be used in 
our main result. The BM algorithm implies that the initial consensus vectors {s,(0) : i <G V} are essentially flooded 
through-out the network. In 0], lfl6ll the general methodology of the BM algorithm is discussed wherein it is referred 
to as "flooding". For completeness of our results, we show in Theorems 14. 214. 3l that regardless of the communication 
pattern between nodes, there exists no distributed algorithm (|2|), ((3]) that can obtain average-consensus before the 
BM algorithm. This is why we have named it the "bench-mark" algorithm. 

Let <$[•] denote the Kronecker delta function applied element-wise, and e; denote the i th standard unit vector in 
K n . The BM signal specification ([3]) and knowledge set update (|7| are respectively defined as (fT0|) and (jl ip below. 



Algorithm 1: Bench-Mark (BM) 

Signal Specification: SFfflrf = < ^oJ\\J^j\oJS j\ o J f » « (1Q) 

{ {vM j )jM)} if^(^') = il„ 

Knowledge Set Update: 

ru a t ^ \ {i,nMA j (+))Ji(t\H+)),smves-t-v?(t\ j ,4 j ) = i} if«i(4 J '(+))^il« M1 , 

M*i (+)) = < (H) 
[ {*(«?(+)),*(<?(+))} if *(*?(+)) - il„ 

Normal Consensus Estimate Update: v^tf (+)) = -v ij (t^ (12) 

I Efci^(*i j (+))*<(°) if *<(*?(+)) 7^ Ll n 
Consensus Estimate Update: Sjfiv (+)) = < (13) 

I *(0) ift) i (^(+)) = il n 

Estimate Initiahzation: 6i(0) = — e; , Sj(0) = — sJO) . (14) 

n n 



In Lemma 17731 and Lemma 17721 of Appendix VII we will prove that (JT75J is the unique solution to (|9|) under (fT0|) 
and (fTT]) . and that (fl4| is the unique solution to (|9]) under the initial knowledge set (A4). The update (ff3|) follows 
immediately from (|12|) and the relation j8]). Notice that the BM algorithm updates (fTT]), (fl2|) together with the 
signal specification (TOl imply s(0) £ /C. i (^(^'(+)) iff = ±1„, and likewise s e (0) e /C i (^(+)) iff 

fiu(ti(+)) = ~ and Uj(^ J '(+)) ^ ~l n . It thus follows that (jTOj), (JTTJ and (jT2]) imply that the consensus estimate 
update Si(t* J (+)) defined in (fT3|) can be locally computed at node i based only on /Cj(tJ') and the received signal 

Besides flooding the initial consensus vectors, the BM algorithm (fTO)) — (fl4"|) has an additional feature that is 
not necessary but is rather practical: if Vj(t) — —l n then all n of the initial consensus vectors are stored in JCj(t), 
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when this occurs the consensus estimate Sj(t) defined in ([D]) will equal s(0) and thus node j has obtained average- 
consensus. In this case all of the initial consensus vectors are deleted from Kj(t), and any signal transmitted from 
j contains only the consensus estimate Sj(t) — s(0) and the vector Vj(i) = Upon reception of a signal 

containing Vj(t) = ^-ln, the receiving node i can delete all of their locally stored initial consensus vectors and 
set §i(t) — Sj(t) — s(0). In this way the average-consensus value s(0) can be propagated through-out the network 
without requiring all n initial consensus vectors to be contained in every signal. See Conjecture 17.41 in Sec lVII-Al 
for a conjecture regarding the BM algorithm. 

B. Algorithm 2: Distributed Averaging (DA) 

The DA algorithm that we now introduce, to the best of our knowledge, is new. To define the DA update procedure, 
let V + denote the pseudo-inverse of an arbitrary matrix V. The DA signal specification ([3]) and knowledge set update 
are defined as JT5]) and Jl6| below. 



Algorithm 2: Distributed Averaging (DA) 






Signal Specification: , t\ j ) = Kj )\{j,n, sj (0) } 




(15) 


Knowledge Set Update: £<(#'(+)) = {», n, «<(*?(+)), *(«?'(+))> *i(0)} 




(16) 


Normal Consensus Estimate Update: u»(ti'(+)) — ^(D4)^M) _ ln > Vp^j = [vi(tY),Vj(t^ 


n 


(17) 


Consensus Estimate Update: sAtV '(+)) = K^n^^ln > K = [s.jft,- 7 ) , s 7 -(tn ), — (0)1 




(18) 


Estimate Initialization: Vi(0) = — , Sj(0) = — Sj(0) . 

n n 




(19) 


The Lemma 17.21 and Lemma 17.51 in Appendix VII, respectively, prove that (fT9)l is the unique solution of (|9|) 
under (A4), and that (|17p is the unique solution to (|9]) under (fT5|) and (fT6)l. Based on the normal consensus update 
(fTTj) the relation J8| reduces to (fT8|l. From (fT~8]) it is clear that the DA consensus estimate update §i(ii (+)) can 



be locally computed at node i based only on JC^tf) and the received signal S 13 (t^ ,tf). Notice that VmA) i s a 
n x 3 matrix, thus the pseudo-inverse in (fT7|) will have an immediate closed-form expression, see (|70|) . (|72|) and 
(|73l) in Lemma [7791 Also note that under the DA algorithm every signal contains only the local consensus estimate 
Sj(t) together with the local normal consensus estimate Vj(t), 

C. Algorithm 3: One-Hop (OH) 

Under the OH algorithm each signal (t l Q , ) will either contain the local initial consensus vector Sj (0) and 
transmitting node identity j, or the average-consensus value s(0) and a scalar to indicate that the transmitted 
vector is the true average-consensus value. For this reason the conditions for average-consensus under the OH 
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algorithm are relatively straight-forward to derive (see Theorem |4.7| in Sec. IV and the proof in Appendix VII). The 
OH algorithm signal specification and knowledge set update are respectively defined by (|20f and (12 ip below. 



Algorithm 3: One-Hop (OH) 








Signal Specification: = U * M} * ^ * » U 

{ {o, «,•(*«)} if%(4 j -) = ii n 






(20) 


Knowledge Set Update: (*?(+)) = < 77 W/ 


if «,(*«(+)) ^ 
if «*(*?(+)) = 


n " 

1 ln 

n ri 


(21) 


Normal Consensus Estimate Update: 








u y(f,^) = l n -^.(^) + e i ] 








(+)) = S , 

\ il„ ifO = S?(tg',t?) 






(22) 


f M4 J )+(£-»>« (*?')) 
Consensus Estimate Update: (+)) = < 

I 5(0) 


ifo^s«(4 j 
ifo .s; -(/;,' 


,4 J ') 


(23) 


Estimate Initialization: v«(0) = — , s,(0) = — Sj(0) . 

n n 






(24) 



The Lemma 17.21 and Lemma 17.201 in Appendix VII respectively prove that (|24|) is the unique solution to ((9]) 
under (A4), and that ((22)) is the unique solution to @ under (j20)) and pip . Notice that ((22|) implies the relation 
([8]) reduces to (|23|) . Given (|20|) . (pT|) and l|22[) it follows that the OH consensus estimate update Si(tfi (+)) defined 
in (|23|) can be locally computed at node i based only on Ki{t % i) and the received signal S^iffi , t^). 

D. Algorithm 4: Discretized Distributed- Averaging (DDA) 

For the DDA algorithm let the discrete set of vectors M™ j_ be defined, 

I°,={i)eM" : vt G {0,1/ra} , V 1 = l,2,...,n }. (25) 

The discretized version of ((9]) that we consider under the DDA algorithm is, 

Vi{t\ J (+))= arg^ min(w- il„) 2 , 

.... (26) 
s.t. © holds and v <= R™ L , given /Q^) U£ y (*o '4 J )' 

To define the DDA normal consensus update it is convenient to denote v~ % £ W 1 ^ 1 as the vector v € 1" with 
element i>j deleted. The DDA signal specification and knowledge set update are defined below, 
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Algorithm 4: Discretized Distributed-Averaging (DDA) 






Signal Specification: S ij (tf ,t\ j ) = \ {j, n, Sj (0)} 




(27) 


Knowledge Set Update: &(#(+)) = U, n, SiiffM), Sj(0)} 




(28) 


Normal Consensus Estimate Update: 






*i(*i 7 (+)) = + &fy(*o ) + ^ 




(29) 


(a,S,c) = (l, 1, -^(4 J )) ifV(4W(4i) = 






(a, M) = ( 0, 1, I - if Or'CtftV^o') > > V(4 J ') 2 


<V(*o') 2 


(30) 


(a, 6, c) = ( 1, 0, 0) if ir(t\ j )%% j ) > , V(4 J ') 2 


>V(*o') 2 




Consensus Estimate Update: Sifti (+)) = a§i(tf) + bsj(t^) + cs 2 (0) 




(31) 


Estimate Initialization: ^(0) = — , s,(0) = — Sj(0) . 

n n 




(32) 



Notice that the DDA signal specification ((27]) and knowledge set update (|28|) are identical to those of the DA 
algorithm. Also notice that under (|29l the relation (|8]) implies pT)) . The Lemma I7T21 and Lemma l7.27l in Appendix 
VII respectively prove that (|32|) is the unique solution to ((26]) under (A4), and that ((29)) — p0|) is a global solution 
to ((26)) under ((27() and (|28l) . From ((27)) and (|28)) it is clear that the DDA consensus estimate update Sj(i^(+)) 
defined in pT)) can be locally computed at node i based only on /Cj(t^) and the received signal (t % Q ,ti). 

Summary: We have defined four algorithms that in Sec. IV will be shown to achieve average-consensus. The 
algorithms were derived as special cases of the distributed algorithm ((3]), (|7j, where ((TJ) is a special case of (j2j- 
Computationally, each algorithm requires only a few elementary arithmetic operations. In Sec lV-AI we discuss the 
storage and communication costs of these four algorithms. 

IV. Main Results: Analysis of the Average-Consensus Algorithms 

This section proves that Algorithms 1 — 4 described in Sec. Ill achieve average-consensus under different communi- 
cation conditions, we state this below in five theorems. Under assumptions (A1)-(A9) listed in Sec lH-AHH-BI these 
theorems provide necessary and sufficient conditions on the communication between nodes for average-consensus 
formation under Algorithms 1 — 4 (recall Definition 12.21 defines the solution to the average-consensus problem). 
The main implication of these results is that, with or without flooding, an average-consensus can be achieved 
in the presence of arbitrary communication delays and link failures, provided only that there is a uni-directional 
connectivity condition among nodes. 

Each theorem below will assume a certain condition on the communication among nodes. To specify these 
conditions we require the following two definitions. For any t\ > t > 0, an arbitrary "communication sequence" 
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C(t ,*i) is defined as the set of all signals transmitted after time to and received before time t\, that is, 

= {S hjl ,S i ^,S i ^,...} 

where we have omitted the time indices but it is understood that the transmission time t e 31 and reception time t 1 ' 31 

of each signal S %tot belong to the interval [to,ti]. Recall that a signal S ij (t^ , t ± ) as stated in (A2) implies that a 

well-defined subset of K.j{t % 3 ) leaves node j at time and is received at node i at time t 13 (the specific subset 

depends on the algorithm considered). 

Next we define the notion of a "communication path". Intuitively, a communication path from node j to i implies 

that node j transmits a signal received by some node t\, and then node l\ sends a signal received by some node 

£2, and then node £2 sends a signal received by some node £3, and so on, until node i receives a signal from node 

£k(ij) - Technically, we say that Cu tl j contains a "communication path" from node j to node i iff Cu 0itl ] contains 
v 

[to(ij),ti(ij)] 



a sub-sequence C\ 3 with the following connectivity property. 



-) r ciu cl?ti 0M2 
u [t mMm - iS ' S ,D ' (33) 

... ? 5*m«A(«)-i } 5»4(«)} 

where again we have omitted the time indices but it is understood that the transmission time t q q of each signal 
gi q+1 e q occurs a fter the reception time t l ^ lq ~ 1 of the preceding signal S iqiq ^ 1 . Note that the communication path 

c tmMm has a cardinalit y I mMm I ^ + L 

A. Necessary and Sufficient Conditions for Average-Consensus 

We are now ready to present the main convergence theorems for Algorithms 1—4. 

1) Algorithm 1 (BM): To prove convergence of the BM algorithm, consider the following communication 
condition. Let us denote V_i = V \ {i} for an arbitrary node i E V. 

Definition 4.1: (SVSC) A communication sequence C[o :tl ] is "singly V-strongly connected" (SVSC) iff there 
exists a communication path from each node i E V to every node j 6 V_i. □ 
We will let "C [t0)tl ] G SVSC" denote that a sequence C[ toM] is SVSC. The definition (J33j) implies that C [t0itl ] G 
SVSC iff, 

C H(ij)Mm cC l^^ Vj€V_ i; V*GV. (34) 

The following theorem establishes the sufficient communication conditions for the BM algorithm (fT0|) — {HJ. 

Theorem 4.2: (BM Sufficient Conditions) Consider Algorithm 1, namely the BM algorithm (jTOj) - CH}. Then 
the average-consensus (|6j is achieved at time t = t\(+) for any communication sequence CWt-j satisfying the 
SVSC condition p4)l . 

Proof: See Appendix VII. ■ 
The BM algorithm implies a "flooding" of the initial consensus vectors {si(0) : i G V} through-out the network. 
There is no other algorithm in the literature that does not use a protocol "equivalent" to the flooding technique and 
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still guarantees average-consensus for every communication sequence Cm jj satisfying ((34)) , we state this formally 
as Conjecture 1741 in Sec lVII-AI Also related to this is Conjecture 17.381 in Sec.VII-B of Appendix VII. 

When combined with Theorem 14.21 the following theorem implies that the communication condition sufficient 
for average-consensus under the BM algorithm is the exact same communication condition that is necessary for 
average-consensus under any algorithm. This is why the BM algorithm can be said to possess optimal convergence 
properties. 

Theorem 4.3: (BM,DA,OH and DDA Necessary Conditions) If a communication sequence CVfj does not satisfy 
the SVSC condition ((34)) , then no distributed algorithm (|2j), <J3]) can achieve average-consensus (|6|) at time t = 

Proof: See Appendix VII. ■ 

Although Theorem 14.31 is somewhat obvious, it is also a valid necessary communication condition and will be 
referred to through-out the paper. Note that Theorem 14.71 below states that the OH algorithm requires even stronger 
necessary conditions than the SVSC condition ((34)) implied by Theorem l4.3l However, the DA and DDA algorithms 
do not; there exist many SVSC sequences under which the DA and DDA algorithms will obtain average-consensus 
at the same instant as the BM algorithm. An example is the "unit-delay double cycle sequence" defined in (1 1701) 
of Appendix VII. Together with Theorem 14.31 the example (11701) implies that the DA and DDA algorithms possess 
the weakest possible necessary conditions for average-consensus that any algorithm can have, this is illustrated in 
Fig Q] below. 

2) Algorithm 2 (DA): The convergence of Algorithm 2 depends on the following IVSC condition. 

Definition 4.4: (IVSC) A communication sequence CWji is "infinitely V-strongly connected" (IVSC) iff for 
each time instant t 6 [0, ti) there exists a finite span of time T t € (0, t\ — t) such that Cu t+ x t ] satisfies the SVSC 
condition (J34j). □ 
It follows that a sequence CWji is IVSC iff we can define the infinite set of ordered pairs {(tQ,t{) : i € N}, 

tg = , 4 = arg min t {C [tU e SVSC} 
4=t{- 1 (+) , € = 1,2,.... 
A sequence Cmtji being IVSC is then equivalent to the condition, 

C[o,ti] = LUn C [t l ,t[\ 



C [t g >tf] e SVSC , V £ G N 



(35) 



We now proceed to the convergence of the DA algorithm. The following theorem is our main result. 

Theorem 4.5: (DA Sufficient Conditions) Consider Algorithm 2, namely the DA algorithm (fT3|) — (fT9)) . Then 
average-consensus (|6| is achieved at time t = for any communication sequence Cm jj satisfying the IVSC 

condition (1351) . 

Proof: See Appendix VII. ■ 
The above result is interesting because the IVSC condition ((35)) assumes a weak recurring connectivity between 
nodes, and also because the resource costs of the DA algorithm are significantly lower than the BM algorithm 
(see Sec lV-AI ). Many papers on average-consensus formation such as flTTJ, 0, IflOl , J26), ll2D . Il25l assume 
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communication conditions that are special cases of the IVSC condition. See also Conjecture 17.391 in Sec.VII-B 
of Appendix VII. 

3) Algorithm 3 (OH): The following SVCC condition will be shown sufficient for convergence of the OH 
algorithm. 

Definition 4.6: (SVCC) A communication sequence Cmjj is "singly V-completely connected" (SVCC) iff there 
exists a node i E V and a time instant tx/2 E (0,ti) such that, 

Sv(ti j ,t\ j )€C lQtty2] , Vjev., 

□ 

The first line in (l36*j) implies that during the interval [0,ti/a] every node j E V_i will have transmitted a signal 
that was received by node i. The second line in (|36t implies that during the interval (fi/2,ti] the node i will have 
transmitted a signal that is received by each node j E V ;. We will let "Cr to t] i € SVCC" denote that a sequence 
Cr t0itl ] is SVCC. Note that the identity of node i need not be known by any node. We now consider convergence 
of the OH algorithm. 

Theorem 4.7: (OH Necessary and Sufficient Conditions) Consider Algorithm 3, namely the OH algorithm (|20[) — 
(|24|) . Then average-consensus © is achieved at time t = ti(+) iff the communication sequence Crg^i satisfies 
the following condition: 

. (C): for each node i for which there exists a node j E V- % such that S lj (t^,tY) i C[o, tl ] for all (4 3 ',*i') G r2 > 
there exists a communication path Cfw^\ tl (j^)i G C[o,ti] from at least one node £ such that (t^ , t 1 ^ ) E 
C[o,t (ie)) for all j e V_£. 

Proof: See Appendix VII. ■ 



Notice that any communication sequence that satisfies the SVCC condition (|36[) will also satisfy the condition 
(C) stated in TheoremgZZJ take 1 = 1, t {jt) = t 1/2 {+), and C j { ^ mMm = S' ]l {t^ for all j e V- ( . This 
relation motivates the position of the OH algorithm in the Venn diagrams presented in FigQ] See also Remark [l. 361 
in Sec lVII-Al The condition (C) is more general than SVCC, it implies that each node i E V will either receive a 
signal directly from every other node j E V_,, or have a communication path from some node I after the node I has 
received a signal directly from every other node j E V_£. We have defined the SVCC condition not only because 
it is sufficient for average-consensus under the OH algorithm, but also because it is necessary for the definition of 
the communication condition IVCC described next. 

4) Algorithm 4 (DDA): The sufficient conditions for convergence under Algorithm 4 requires the following 
definition. 

Definition 4.8: (IVCC) A communication sequence C[o,ti] is "infinitely V-completely connected" (IVCC) iff for 
each time instant t E [0, ti) there exists a finite span of time T t E (0, t\ — t) such that C[ t ,t+T t ] satisfies the SVCC 
condition l(36j), □ 
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It follows that a sequence CW^i is IVCC iff we can define the infinite set of ordered pairs {(io,ij) : £ G N}, 

fg = , 4 = arg min t {C*^ t] G SVCC} 
t t o = tt 1 (+) ,1 = 1,2,.... 
A sequence Cmtjj being IVCC is then equivalent to the condition, 

C[o,ti] = UeeN C [t e ,t{] . 
Cfat*] G SVCC , V I G N . 

We note that the specific node i can vary between each Cj t < sequence, and furthermore we do not assume that 
any node knows the identity of i. The following theorem deals with the convergence of Algorithm 4. 

Theorem 4.9: (DDA Sufficient Conditions) Consider Algorithm 4, namely the DDA algorithm (|27j) - l[32|). Then 
average-consensus Q is achieved at some time t G (0, ti) for any communication sequence CVij satisfying the 
IVCC condition (|37| . 

Proof: See Appendix VII. ■ 
The above result is not quite as interesting as Theorem |43J since even though the DDA algorithm is a discretized 
version of the DA algorithm, the IVCC condition ([37]) is far stronger than the IVSC condition (|35|) . Also observe 
that the OH algorithm obtains average-consensus for any C[ tl ] G SVCC, whereas the DDA algorithm will obtain 
average-consensus under the much stronger condition that Cm tl i satisfies (|37[) . However, as mentioned above, 
due to example (I170[) of Appendix VII the DDA and DA algorithms both can obtain average-consensus under 
suitable SVSC sequences, thus they possess much weaker necessary conditions than the OH algorithm. In fact, any 
communication sequence Cm tl i that strictly satisfies (|36| implies Algorithms 1 — 4 all obtain average-consensus at 
the exact same time instant. This is noteworthy because many past algorithms can only achieve average-consensus 
asymptotically (e.g. most iterative averaging schemes), in contrast all four of the algorithms considered here can 
achieve the (finite) bench-mark time for average-consensus under appropriate communication sequences (e.g. any 
sequence CV^i that strictly satisfies (f36|) >. 

Summary. Theorems 14. 2[|4.3[|431l4. 7114.91 above state necessary and sufficient communication conditions for 
average-consensus under Algorithms 1—4 given the assumptions (A1)-(A9). Each theorem is associated with 
one of four connectivity assumptions on the communication sequence CV^j, denoted by { SVSC, IVSC, SVCC, 
IVCC } defined in f34] ) .(|33 ]) .(|36 ]) . and (J37j) . Observe that assumptions IVSC and SVCC are sufficient conditions for 
SVSC. Furthermore, IVCC implies that both IVSC and SVCC are satisfied, see FigQ] Notice that each connectivity 
condition assumes a set of directed signals with an arbitrary delay in the transmission time of each signal. This 
is significant because, apart from the flooding technique, no other consensus protocol in the current literature can 
ensure average-consensus in the presence of arbitrary delays in the transmission time of each signal for a priori 
unknown communication sequences. Of course if the communication sequence is known a priori, then specific update 
protocols can always be constructed that guarantee average-consensus at the same instant as the BM algorithm. 
Current results on average-consensus that do allow communication delays assume the delays have pre-determined 
upper-bounds, and also require the use of averaging weights that are globally balanced and pre-determined off-line 
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(see for example |2~T1 . |fT31 , [28 1). On a related note, besides flooding there appears to be no consensus protocol 
in the literature that has been proven to guarantee average-consensus for every communication sequence CW-j 
satisfying any one of the conditions (|3"4"l) . (f35j) . (|36[) . or (1371) . The majority of past results on average-consensus 
either require special cases of the IVSC condition, or else can only guarantee approximate average-consensus under 
the SVSC condition fl9l , 0, (TJ. On the other hand, the two non-trivial algorithms DA and DDA require a set 
with cardinality upper-bounded bounded by 0(n + d) to be communicated and stored at each node, and all previous 
algorithms besides the flooding and randomized protocols do not possess this drawback. 

The Venn diagrams in Fig[T] summarize Theorems I4.2H4.31|431|4.7H4.9I as well as their relation to the Gossip 
and ARIS algorithms that will be used as comparisons to the four proposed algorithms. It remains an open problem 
whether any protocol exists that guarantees average-consensus for all IVSC sequences without requiring a set with 
cardinality upper bounded by 0(n + d) to be stored and communicated at each node. 



SVSC 

(BM), (DA), (DDA) 



/TVSC 


/ rvcc\ 


svcc\ \ 


/ /TVSC 


/rvcc\ 


svcc\ 


( (ARIS) 


\ 




/ (ARIS) 






', (DA) 


(DDA) J 


(OH) 






(OH) J 



Gossip 



Gossip 



Sufficient Conditions 



Necessary Conditions 



Figure 1. Venn Diagram of Sufficient and Necessary Conditions for Algorithms 1-4 as well as the Comparison Algorithms Gossip and ARIS 
to Achieve Average-Consensus. The condition (C) in Theorem I4.VI is omitted for simplicity of presentation. 



V. Numerical Implementation and Examples 

This section presents the communication and storage costs of the various algorithms proposed above. In Sec lV-AI 
below we define the resource costs for the six algorithms compared in this paper (four proposed in Sec. Ill and two 
others in Sec.VII-B). In Sec lV-BI we present numerical simulations of the six algorithms under various randomized 
full network graphs when assuming instantaneous communication. 

A. Algorithm Resource Costs 

Each of the four average-consensus algorithms as well as the two comparison algorithms presented in Sec.VII-B 
require that the knowledge set ICi (t) and signal set (t 1 ^ , t( ) are respectively defined by a set of scalars, where 
each scalar has a particular meaning. We are thus motivated to quantify the "resource cost" of each algorithm 
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Algorithm 


BM 


DA 


OH 


DDA 


Gossip 


ARIS 


min(0i(t)) 


4d + 5 


4d + 6 


4d + 5 


4d + 5 


2d 


7 + 2(r + 2)d 


max(0 4 (i)) 


2nd + 4 + Lf J 


4d + 2n + 4 


4d + 4+ Lf J 


4d + 4+ Lf J 


2d 


L|J + 6 + 2(r + 2)d 


mm(pij (t)) 


2d + 1 


2d+ 1 


2d + 2 


2d+ 1 


2d 


3 + 2(r + l)d 


max(pij(t)) 


2(n-l)d+LfJ 


2d + 2n 


2d+ 2 


2d+LfJ 


2d 


L|J +2 + 2(r + l)d 


min(wi(*)) 


6d + 6 


6d + 8 


6d + 7 


6d + 6 


4d 


10 + 4(r+ |)d 


max(cj i (t)) 


2(2n- l)d + 4 + 2Lf J 


6d + 4n + 4 


6d + 6+ L§J 


6d + 4 + 2Lf J 


4d 


2Lf J +8 + 4(r + |)d 


O(0) 


nd 


n + d 


n + a! 


n + d 


d 


rd + n 


O(p) 


nd 


n + d 


d 


n + d 


d 


rd + n 



Table I 

Minimum and Maximum Resource Costs obtained by applying each Algorithm in Sec. Ill to J38j, 1391) , and Boll . Here d 

DENOTES THE DIMENSION OF THE INITIAL CONSENSUS VECTORS, n DENOTES THE NUMBER OF NODES, pij (t) DENOTES THE SIGNAL SET 
DIMENSION, <f>i(t) THE KNOWLEDGE SET DIMENSION, AND O)j(t) IS THE SUM OF Pij(t) AND </>;(t). WE LET [-J DENOTE THE "FLOOR" 

OPERATION. 



in terms of the total number of scalar values that are required to define the two sets /Cj(t) and S 1 ^ (t^ , ) . In 
particular, for any t > we define the "storage cost" of an arbitrary node i E V as <pi(i), 

<t>i(t) = the minimum number of scalars 
V ' (38) 
required to define the knowledge set /Cj(t). 

Likewise we define the "communication cost" pij{t l (l) of an arbitrary signal S i i (iff , t | J ) as follows, 

Pa(tn) = the minimum number of scalars 

(39) 

required to define the set S 1 ^ (t 1 ^ , t± ) . 
Next define the total resource cost of node i at time t > as uii(t), 

Wi(t) = &(*) + Pij(t) ■ (40) 

Based on the knowledge set and signal specifications defined in Sec. Ill, the Table I presents the resource cost 
computations of each algorithm when using the definitions (|38[) . (|39|) , and (|40|) . The entries of Table I are derived 
in Sec lVII-Cl of Appendix VII. 

Note that the storage cost 4>i(t) is defined per node, and the communication cost pij(t) is defined per signal. 
The total maximum resource costs of the BM algorithm increase on the order 0(nd), whereas the total maximum 
resource costs of the DA and DDA algorithm increase on the order 0(n + d). Although the total maximum 
resource costs of the OH algorithm increase on the order 0(n + d), the maximum resource cost of each signal 
under the OH algorithm increases only on the order 0(d). However, the communication conditions necessary for 
average-consensus under the OH algorithm are much stronger than the conditions necessary under the BM, DA, 
and DDA algorithms. This disparity makes it difficult to state definitive results regarding the least costly algorithm 
under general communication sequences. If condition (C) in Theorem 14.71 is known to hold a priori, then the OH 
algorithm may be preferable to the DA and DDA due to the lower communication costs 0(d) < 0(d + n). On 
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the other hand, under example (|170|) in Appendix VII the DDA algorithm is preferable to both the DA and OH 
algorithm, since the former implies larger resource costs and the latter will not obtain average-consensus. 

The Gossip algorithm in J3) has total resource costs that increase on the order 0(d), however this algorithm 
requires strictly bi-directional and instantaneous communication, see Sec.VII-B as well as Figf2] in Sec lV-BI The 
total resource costs of the ARIS algorithm increase on the order 0(rd+n), where r is an ARIS algorithm parameter 
explained in Sec.VII-B. If r > n then ARIS is more costly than the BM algorithm. For this reason, the simulations 
presented in Sec lV-B| assume r = n. The RIS algorithm proposed in [19| requires r random variables to be initially 
generated at each node for each element of the respective local initial consensus vector Sj (0) € M. d , the RIS algorithm 
also requires these random variables to be communicated between nodes, thus both the storage and communication 
costs of the RIS algorithm increase on the order 0(rd). 

B. Numerical Results 

We present here numerical simulations of the four proposed algorithms together with the two comparison 
algorithms. The algorithm parameters were chosen as n — 80 (number of nodes), d — 1 (dimension of the 
initial consensus vectors), r = n (number of ARIS random variables generated per initial consensus element), and 
Si(0) = i, V i E V (initial consensus vector values). The node communication is assumed to be instantaneous and 
in discrete-time, using the following randomized protocol: 

• at each k E N, two nodes E V 2 are uniformly chosen at random such that i ^ j, 

• with probability one, the node i sends a signal to the node j at time k, 

• with probability p the node j sends a signal to the node i at time k. 

We compare four choices of p, namely p E j,0}. Note that p = 1 implies instantaneous bi-directional 

communication. As p decreases there will be fewer expected signals per time instant, thus we expect that each 
algorithm will have slower convergence for lower values of p. 

Fig|2] shows the network consensus error under each algorithm for each value of p G {1, |, j, 0}. It is clear 
that the Gossip algorithm only converges to average-consensus for p = 1. For p E {i i 0} the Gossip algorithm 
converges to a consensus that is increasingly distant from the average-consensus. The BM algorithm converges 
fastest in all simulations, as expected from Theorems 14.2114.31 Initially the DA algorithm can be observed to 
almost match the BM algorithm in all simulations, however as time proceeds for p — 1 the Gossip and ARIS 
algorithm eventually overcome DA, and in p E {h, |,0} the ARIS algorithm eventually overcomes DA. However, 
the resource costs of ARIS in our simulations is greater than even the BM algorithm, thus further research is needed 
to objectively evaluate the trade-off between resource cost and network consensus error for each algorithm under 
various communication conditions. 

VI. Conclusion 

This paper has described and analyzed four distributed algorithms designed to solve the average-consensus 
problem under general uni-directional connectivity conditions. We derived necessary and sufficient conditions for 
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Figure 2. Convergence of Consensus Algorithms for Communication Probabilities p S {1, |, ^,0} (top left to bottom right). The consensus 
error E is defined above Observe that the Gossip consensus error vanishes only for p = 1, because this is the only model that implies 
strictly bi-directional communication. All algorithms converge moderately slower as p decreases. 



the convergence to average-consensus under each respective algorithm. The conditions for convergence were based 
on two types of connectivity conditions, namely the singly V-strongly connected sequence (defined in Sec lIV-AT|) , 
and the singly V-completely connected sequence (defined in Sec lIV-A3V Both connectivity conditions allow arbitrary 
delays in the transmission time of each signal, and we did not assume that the sending node knows the identity of 
the receiving node. The resource costs for each of the algorithms were derived and shown to differ in regard to their 
order of magnitude with respect to the parameters n (number of nodes) and d (dimension of the initial consensus 
vectors). Comparisons were made with two known consensus algorithms, referred to as the Gossip algorithm and the 
adapted randomized information spreading (ARIS) algorithm. Simulations were provided as well as Venn diagrams 
of the connectivity conditions required for average-consensus under each algorithm. The non-trivial algorithms 
considered here are relatively advantageous under weak communication conditions if the dimension d of the initial 
consensus vectors exceeds the network size n. The works [4| and ll20l provide two practical examples of when 
d > n might typically be the case, e.g. distributed inference regarding a ^/n, dimensional process or parameter in 
noise. 

The four communication conditions we proposed were deterministic; there were no stochastic properties assumed 
in regard to the signal process between nodes. However, our framework allowed directed signals as well as arbitrary 
delays in transmission time, hence every causal signal sequence is a special case of our framework. This suggests 
that the four proposed algorithms can be applied to stochastic communication models for which there is a non-zero 
probability of consensus under any distributed algorithm, however future work is needed in this direction. 

Future work is also needed to obtain a lower bound on the convergence rate to average-consensus for given 
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characterizations of the communication sequence under each proposed algorithm, as well as improved algorithms 
designed specifically for particular communication sequences. For an objective evaluation of the various average- 
consensus algorithms, additional research is needed to compare the evolution of a cost function of the resource cost 
and network consensus error under a variety of communication sequences. Lastly, an interesting generalization of 
the average-consensus problem is to allow the initial consensus vectors Sj(0) to vary with time, as discussed for 
instance by |20|, |8i|, 11221 . Q3|, |29|, [16]. Applying the algorithms and communication conditions proposed in this 
work could yield further results with regard to ensuring the average s(t) is contained in the knowledge set 
of each node i g V at some time instants t for given dynamic models of the set {si(t) : i E V}. 

As a final note, it is worth mentioning that each of the four proposed algorithms can obtain a consensus on any 
linear combination of the initial consensus vectors {si(0) E W 1 : i E V} under the exact same communication 
conditions as stated in the main results. In other words, suppose a vector w E K" was initially known at each node, 
then if each algorithm updated the normal consensus estimate based on |9]) with -^l n replaced by the vector w, 
the same conditions stated in Sec. IV will imply the respective algorithms ensure Sj(ii(+)) = X^iev s i$) w i f° r 
all i E V. The proofs of these results follow by identical arguments to those presented in this work, simply by 
replacing il„ by the vector w. 



In Sec lVH-Al of this appendix we derive the consensus estimate initialization for each of the four proposed 
algorithms, as well as the proofs for Theorems 14.214.314.514.71 and 14.91 In Sec lVH-Bl we define the Gossip algorithm 
proposed in [3| in terms of the class of distributed algorithms (|3|), (0, and then we define the ARIS algorithm in 
these terms as well. In Sec lVII-Cl we derive the resource costs presented in Table I of Sec lV-AI and in Sec lVII-Dl 
we define the "unit-delay double cycle sequence" as an example of a SVSC sequence that implies the DA, DDA, 
and BM algorithm all obtain average-consensus at the same instant. We present two conjectures in Sec lVH-Al and 
two conjectures in Sec lVII-Bl 

Through-out the appendix we denote the "error" of the normal consensus estimate ii (t) as, 



VII. Appendix 




(41) 



The total reduction in normal consensus squared error resulting from the sequence C[ to tl ] is then, 






(42) 



S'Ht' 6 i ,tY)ec [ . 



where E 2 (t l Q , tf ')) is defined using the normal consensus error E^ ln {v(t)^) in (|4T| . 




(43) 



It is convenient to use the following definition. 
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Definition 7.1: Under ([8]), ©, and ©, the average-consensus problem is solved at time t iff the "network normal 
consensus error" 5ZT=i -^-i„ (^»C0) * s zero > that is, 

n 

Y,Ei ln (vi(t))=0 , (44) 

i=i 

where Ei 1 n (ii(t)) is defined in (|4T|) . 

A. Algorithm Convergence Proofs 

Lemma 7.2: (Consensus Estimate Initialization) Given the initial knowledge state (A4), the unique solution to 
both Q and (O is, 

fii(0) = -e t , s,(0) = - 8i (p) . (45) 
n n 

Proof: Under (A4) the update © becomes, 

Vi{0)= argg min (v - ±l n ) 2 , 

(46) 

s.t. © holds, given 1Q(0) 2 {i,n, Sl (0)}. 
Observe that the vector a can be locally constructed at each node i E V based only on the data {i, n}. Next observe 
that Sj(0) = Set for all i € V. Given that s,(0) = Se^ is known by node i at t = 0, the set of vectors Wj(0) for 
which © holds is spanje^}, thus (|46|) becomes, 

fii(0) = arg^ min- g span { e J (« - ^nf , 

= VV+±l n , 7=[ 6i ] (47) 
= V(V'V)-'V'H n = ie, . 

If 0i(0) = ^e, then © implies Sj(0) = ^Si(O). Note that ie, and ^Sj(O) can both be initially computed at node 
i given that /Q(0) 2 {i,n, Sj(0)}. The solution (|45l) implies £Sj(0) G M™ j_ and thus (|45j) is a feasible solution to 
(|26j) . It then follows that (|45j) is also the unique solution to (|26j) since the feasible space of ((26)) is contained in 
the feasible space of ((9]), and both © and (|26|) minimize the same objective function. ■ 
Lemma 7.3: (BM Normal Consensus Estimate Update) Applying (fT0|) and (jTTJ to © yields the BM normal 
consensus estimate update (fT~2|) - 

Proof: If Vi(tY) = —l n then © implies Si(if') = s(0) and thus node i need not update its knowledge state 
regardless of the signal 5^'(*o .*l )• If fy(*o ) = TT 1 ™ then COS) together with © implies, 

S*(tf,t?) ^(tf), (48) 
3 5(0) . 

In this case © can be re-written, 

= ar gy min (w- ^in) 2 , 



s.t. ® holds, given Sj(t%) = s(0) 



(49) 



Since Sj(t l Q) — s(0) is known we can let •§i(if'(+)) = s(0) and thus obtain = — l n as the unique 

global solution to (|49|) (note that il„ can be computed since (A4) and (A7) imply 1Q(tY) 2 {"})■ Notice that 
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this solution coincides with (fT2| . Next assume that Vi(ii) ^ -l n and Vj(t l g) ^ ^ln- Under (fTO)) and (fTT|) we 
can then re-write (J9j) as, 

"*(*i(+)) = ar g£ mill (-D - il n ) 2 , 

s.t. © holds, given s*(0) = Se £ , (50) 
for all : v?($ t 1$) = 1}. 

Given that s^(0) = Se^ for all {£ : vV [ff , t l Q ) = 1} is known, the set of vectors £>i(tj' 7 (+)) for which j8j holds 
is span{e£ : vV{t % ^,t^) = 1}, thus (|50|) becomes, 

*(*?(+)) = arg^ min. e span{ ^ ; ^ ( ^y ) = 1} («-il») 2 , 

= V#(BM,iln, (51) 

V {BM) = [e t : »j , '(t?,tg'') = l] • 

Let L = u' J '(t] J ,io')'l» denote cardinality of the set : t^- 7 (t | J , t q 7 ) = 1}. Since the columns of Vrsjvr) are 
linearly independent, the right-hand side (RHS) of ([51")) can be computed as, 

= ^V( BM )(/l x l) _1 1^ , 

where Ilxl denotes the identity matrix of dimension L. ■ 

Theorem |4.2[ BM Network Convergence to Average-Consensus) 

Proof: By the BM update (fl2| when any node receives a signal £? y (tg 7 , tj 3 ), the receiving node i either 
receives the desired average-consensus value s(0), or receives the initial consensus vector Sf(0) of every node that 
has a communication path to node j within the time span [0,£q J ). If Cro.ti] satisfies (l34l) then every node has a 
communication path to every other node within the time span [0,ti], thus (A7) and (|12[) together imply that at 
time ti(+) every node i will compute = ~ln- This implies (|44|) holds at and hence by Definition 

(|7.1[) a network average-consensus is obtained at time t = ■ 

Theorem l4JtBM.DA.OH and DDA Necessary Conditions) 

Proof: If CW tl i ^ SVSC then there exists a node i E V that does not have a communication path to some node 
j G V-i within the time span [0, ii]. At time t\ the node j thus cannot have any knowledge that is contained in 
K.i(t) for any t < t\, regardless of the knowledge set update rule (|2j and signal specification ([3]). Hence Sj(ti(+)) 
cannot be a function of Sj(0) for an arbitrary initial consensus vector Sj(0). It then follows that no distributed 
algorithm (O,© can imply (|6j is satisfied at time t < fi(+) for an arbitrary set of initial consensus vectors 
{s,(0) : ieV}. ■ 

We now present a conjecture regarding the BM algorithm. From Theorem 14. 21 and (fTU| — (fT4"l) it follows that the 
BM algorithm implies the following property (P) of each knowledge set Ki(t): 
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. (P): 












^))2{*i(0)} 
-)) 3 {5(0)} 


, if 3 £ s.t. C^ o{ie)Mie)] C [0:tl] 



The condition (P) forms an equivalence class among all algorithms A defined by ((2]) and ((3|). From the second 
line in (P) it follows that any algorithm A that satisfies (P) will imply (j6]) holds at time t = ti(+) for any 
communication sequence C[o_ tl ] satisfying t|34|> . We now conjecture that if an algorithm A implies Q holds at 
time t = ti(+) for every communication sequence C[ tl ] satisfying (l34l) . then algorithm A must have resource 
costs at least as great as any algorithm A that satisfies (P). 

Conjecture 7.4: If an algorithm A guarantees |6]) holds at time t = £i(+) for every communication sequence 
Cjo.ti] satisfying (|34|. then the algorithm A will require that a set with cardinality upper bounded by at least 0{nd) 
can be communicated and stored at each node. 

The above conjecture implies that any algorithm A that satisfies (P) will require that a set with cardinality upper 
bounded by 0(nd) can be communicated and stored at each node, this is why searching for less costly algorithms 
is of importance. The problem is, less costly algorithms tend to require stronger communication conditions than the 
BM algorithm, and they also do not guarantee average-consensus is obtained as quickly as the BM algorithm. We 
note that due to the resource costs associated with the RIS and ARIS algorithms, the Conjecture 17 . 3 8 1 in Sec lVII-B2l 
does not contradict Conjecture 17.41 

Proof. (Theorem 14. 51 l Lemmas |7 .5 1 - 17 . 1 9l Overview of Proof. To prove Theorem 14.51 we initially show in Lemma 
17.61 that the update (|17[) implies each normal consensus estimate ii(t) satisfies the normalization property (l54l) . 
The Lemma 17.91 proves that each normal consensus estimate Vi (t) also satisfies the "zero local error" property 
vu(t) = 1/n. Without the latter, the following lemmas would still imply convergence of all consensus estimates, 
but the final consensus value would not necessarily equal the average s(0) defined in (Q~|). The essence of the 
convergence proof is that the reduction in error that results from any signal will eventually vanish if Cjo^] £ IVSC, 
see Lemma 17. 131 Applying this result to the DA lower bounds on the reduction in error derived in Lemmas 17.101 and 
17.111 we can show that each normal consensus vector will necessarily converge to a common vector, see Lemma 
17.151 Together with the two technical results derived in Lemmas 17.171 and 17.181 we then combine the triangle 
inequality and the "zero local error" property to prove that the common vector will approach in the L 2 norm 
as time approaches t\. This implies (|44[) holds at t\(+) and hence by Definition (|7.1|) a network average-consensus 
is obtained at time t = 

Lemma 7.5: (DA Normal Consensus Estimate Update) Applying (fT3j) and (fTS]) to (j9j) yields the DA normal 
consensus estimate update ^(t^ 7 (+)) defined in (fT7|) . 
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Proof: Under (fT5|) and (fTS"]) we can re-write |9]) as, 

«t (*?(+)) = ar §5 min TT 1 ") 2 . 

s.t. ® holds, given Sj(4 J ) = Sw,(tf ), (52) 
i s ($) = S«,(^'), and a,(0) = Se*. 

Given that Si{££) — Svi(t 1 ^), Sjit 1 ^) = S{ij(tQ J ), and Si(0) = Se^ are known, the set of vectors Vi(t^ (+)) for 
which (|8|) holds is span{uj(t^), Vjit^), e,}, thus (l52j) can be re-written as 

1 2 

= ar Su min In) ■ (53) 

The update (JTVj) follows immediately from (|53l) . ■ 
Lemma 7.6: (DA Consensus Estimate Normalization) Every normal consensus estimate Vi(t) satisfies 

^{tf = ^Vi(t)'l n , V i e V , V * > . (54) 

Proof: Note that ^(O) = ^e, satisfies (|54| for each i S V. Next observe that under (A7) the estimate 
will not change unless a signal S lJ '{t l Q , ) is received at node i. If a signal is received then by Lemma 1731 the 
estimate Vi(t) is updated to the unique solution of (1531) . Thus to finish the proof it suffices to show that if a vector 
v £ ffi™ does not satisfy (|54|) then the vector v is not the solution to (|53j) . To prove this we show that if (|54|) does 
not hold then the vector w defined, 

w = v(y'l n ) /(nv 2 ) , 

will satisfy the inequality 

(w--l n ) 2 < (v--l n ) 2 . (55) 

n n 

Notice that since w is contained in span(?j), the inequality (|55j) implies that v is not the solution to (|53[) . Next 
observe that if a vector v does not satisfy (|54|) then, 

(u 2 - -v'l n ) 2 > . (56) 

Expanding (j56| yields, 

{v 2 ) 2 -2-v'l n v 2 + (-v'l n ) 2 >0 . (57) 

Re-arranging (|57|) then implies (|55l) . 

(« a ) a -2±tA„t; a >-(It;'l n ) 2 , 



2Vi B + i> (^)V_ 2 Ki^! 

n '* n V nu^ / 7i z v z 
2 



v ri / v m; n 

■ 

Lemma 7.7: (DA Non-Decreasing Normal Consensus Estimate Magnitude) Each magnitude Vi(t) 2 is a non- 
decreasing function of t > for all ieV. 
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Proof: Note that under (A7) the estimate Vi{t) will not change unless a signal is received at node i. If a signal 
S' lJ (t l Q , t % l ) is received then the DA update problem (|53[) implies the update v; L (t l ^(+)) must satisfy, 



±i n ) 2 < (w - H n y , (58) 

Vice span {vi(t¥),Vj(t*J),ei}. 



Since {■Di(tf'), Oj^q 3 )} € span {^(tj 7 ), Vj{io), e^} the inequality (|58|) implies, 



(*.•(*?(+)) - ^1«) 2 < min{(^(tf) - il„) 2 , 

(%(4 j )-iln) 2 }. 



Next observe that if a vector w G K™ satisfies (|54|) then, 

(t'- 1 !™) 2 = - 2i V 'l„ + i , 

V n IL / n ' b n 5 (60) 

= - — i; 2 . 

n 

Due to Lemma [7~6l all normal consensus estimates satisfy (|54| , thus we can apply (|60| to ([59)) and obtain, 

^ - W(+)) 2 < min{I - , i - ^") 2 }. (61) 

Subtracting both sides of floTl) from i then yields, 

W(+)) 2 >maxR(^) 2 , 

(62) 

> W) 2 , 

thus each magnitude Vi{t) 2 is a non-decreasing function of t > for all i E V. ■ 
Lemma 7.8: (Equality of Normalized Linear Dependent Vectors) If two linearly dependent vectors ii, Vj € M" 
both satisfy (|54|) . then = v[vj = vf. 

Proof: If Vi and £j are linearly dependent then there exists some k ^ such that ii — kij, thus if both vectors 
also satisfy (|54]) then, 

I^l„ = fci^.l n fc=(^l„)- 1 (^l n ) 

= « 2 /C> 2 , (63) 
vf = k 2 v 2 =>• k 2 = vf/v 2 . 
Combining the RHS of the first and second lines in (|63"T) implies k 2 = k and thus k = ±1 since k ^ 0. We then 
obtain fc = 1 since vf /v 2 > 0, thus ii = ij and the result follows. ■ 
Lemma 7.9: (DA Local Zero Error Property) Every normal consensus estimate £>i(i) satisfies 

Vidt) = -,VieV,Vt>0. (64) 
n 

Proof: By Lemma 17721 iu(0) = — for each i G V. Next observe that under (A7) the estimate ii(t) will not 
change unless a signal (£q ,t l f) is received at node i. If a signal is received then i>i(t) is updated to the solution 
of (1531) . We now show that under the assumption (|64|) the solution Ui(ix'(+)) to (|53j) will imply (|64l) for every 
set vectors {t)j(^), Vjlfi^), ti). First assume Vi(t 1 ^), Vj(tg), and are linearly dependent. In this case the update 
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1 (65) 



problem (|53)1 reduces to the RHS of (|47|) and thus implies (|64[) . Next assume that any two vectors in the set 
{Vi(ii ) , Vj (tQ ) , ei\ are linearly dependent. In this case ([53]) reduces to 

(+)) = arg£ min~ g span { ej;W } (« - ±1» 

where u g {^i(^i'); %(ic/)} * s linearly independent of e». The objective function in (|65|) is 

/(a, 6) = (aie l + il„) 2 , 

= a 2(l ) 2 +6 2 u 2 + 2 6 1 , (66) 

-2aie'il„ - 2bv'±l n + 1 . 

n i n n n n ' n 

The Lemma [7761 implies that v satisfies (|54|) . Note also that ^e^ satisfies (|54|) . thus the objective function (|66|) can 
be simplified, 

/(a, b) = (a 2 - 2a)(i ei ) 2 + (6 2 - 2&V + 2ab-e[v + i . 

v n n n 



The first-order partial derivatives of f(a, b) are, 

2^ = 2( a -l)(Ie i ) 2 + 2&±eJ« > 
%*l=2(&-l) V 2 + 2aie^. 
The second-order partial derivatives of /(a, 6) are, 

a 2 /(a,fe) _ o/i c .\ 2 



(67) 



72- 



d 2 f(a,b) = d 2 f(a,b) = g i , 
dadb dbda n i 

The determinant of the Hessian matrix of /(a, 6) is thus, 

\H{f{a,b))\=2{{-e i fv 2 -(-e' i vf) . (68) 
n n 

Since ie, and t> are linearly independent, the determinant (|68"1) is strictly positive by the Cauchy-Schwartz inequality. 
This implies the Hessian matrix H[f(a,b)) is positive-definite, thus setting the RHS of (|67|) to zero and solving 
for (a, 6) yields the unique optimal values (a, 6), 

« 2 ie'(i ei -w) - (± ei )V(u- -e z ) 

n 1 ri ^ v n L/ ^ n L/ (69) 



From (|69|) the unique solution to ()65l) is thus obtained, 

M E i l+JJ - (i e .)2 l ,2_ ( i e /„)2 „e 4 
" i (ifj)Vl^zifl) - 

Based on (|70| the element ^(i^^)) can be expressed, 

««(#(+)) - "" e '^ e '~") 



(70) 



( — ei) 2 v 2 — ( — e'v) 2 n 

"T(i ei )2„2_(A e / l ,)2 ^ ' 



(71) 



DRAFT 



28 



Note that the last equality in (|7T|) follows since Lemma 17.81 implies v 2 ^ vf under the assumption that v satisfies 
(|54| and is linearly independent from iej. Next assume that Vi(tf), Vj(t l Q) and are linearly independent. In 
this case the solution (fT7| can be expressed, 

= ^ ) (^ ) ^ (DA) )- i v ( ' DA) ii n , 

V(23A) = [Ui(«l J ) I Vj(*o') I 7T e i] ■ 

For notational convenience we denote v$ — Viifil) and Vj — Vj(t l Q). Note that (V/ da ->V(dA)) has the inverse (|72]l 
below, 



( V {DA) V {DA)) 1 



,J i u 3 
r.2 



— Vu \ 

n Li 

— Vjj 

n J L 

J 

n z V 3 

















(v 2 v 2 - 



6, 2 l> 



2\ 



/ 



(72) 



where the determinant | V/ DA \ V( £»a) 1 can be computed as, 



-V 2 V 2 — V 2 V 2 ) 



Right-multiplying ((72|) by V{ DA ^l n and then left-multiplying by the first row of V^dA) yields Vuifil (+)), 

««(#'(+)) = i ^ A) V (D x)i ^W^ - 

+vjv H V 2 t - vlVjvjVji + ViVjVji - v 2 u v 2 v ot 
+ ±vuv ji v' i v j - -vfvji + vfvivjVji - vfvftu 
+v ll v'-v J v 2 - vfvpji + \v 2 v 2 - i(v-%) 2 ) , 

= |V?JV)| 7(^-W a 



(73) 



+2v li v 3i v' i v J - v%%1 



v 2 v 2 - 



The last line in (|73l) follows since | V/da) ^(da) I is non-zero if e^u, and Uj are linearly independent. ■ 
Lemma 7.10: (DA Lower Bound On Signal Reduction in Error) Upon reception of any signal (t^ , ') the 
decrease in the updated normal consensus squared error has the following lower bound, 

^ ln («<(*?'))-^l lB («i(*?'(+)))> 

maxft^') 2 -^^') 2 , (74) 

n(«i(^)'(0 J -(tS'')- «<(*?)))'}• 
fW/- By Lemma |X6l we can apply (|60| to the left-hand side (LHS) of (|74|) , 

£ii n («i(*l J ')) ~ E li „(^(*i J '(+))) =*i(*?(+)) 3 -«<(*?) a - (75) 
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Applying the first line of (J62J) to the RHS of (|75J) then yields, 



>max{0 , %(4 i ) 2 - Vi(t\ j ) 2 } 



(76) 

>%(4 J ') 2 -^(4 J ') 2 - 

Next observe that for any two vectors H%,Vj, 

span {vuVj} C span e;} . 

It thus follows that, 

£2L ln (arg£ min~ £ e . } (5- il„) 2 ) (??) 

< ^Aijargg min~ g span {^ ;{)j } (5- ^l n ) 2 )• 
Let Vi = Vi(tY), Vj = Vj(t L Q) and Vi(t l f (+)) = #»(+) for notational convenience. From (|77j) we have, 

B |i n («0--Bii n (*i(+)) 

;ar g{) min~ g span^^^.} (« - Trin) 2 ) 

= w 2 — V 

where the last line holds due to Lemma [7761 and w is defined, 

w = arg 5 min~ £ span {^ (« " ^™) 2 , 

= arg 5 = a #. + ^ min^ &) (afij + bVj - i±„) 2 . 



> ( arg,~ min~ n ».i (u - ^l„) z )" - w 2 , 

(78) 

2 _ .",2 



Note that Lemma 17.71 together with the initialization (|45|) implies and ij are non-zero. Next assume that Vi and 
% are linearly dependent. In this case (|79|) reduces to, 

*" = ar S6 min w G spanfa} ( 5 ~ ^™) 2 > 

1 nvj ' 



where the last line follows due to Lemma 17.61 Applying (|80|) to (|78|) implies, 

(81) 



vf - vf 



, 



= II (ft — ViV 



1 - 



where the last line follows by Lemma 17781 since Lemma 17761 implies both Uj and vj satisfy (|754)l . and we are assuming 
Vi and % are linearly dependent. 
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Next assume Uj and vj are linearly independent. In this case (|79|) can be solved analogous to (|70|) based on the 
optimization problem ((65]) . 



w = arg 

-2 



rg£ = av % + bvj min (a, b) ^ + W i " 7T 1 ") 2 
Substituting the second line of (|82|) for u) in the third line of (|78[) then yields, 



(82) 



(83) 



A(u/l„-^l n ) 

— n V «i«|-(«ii)j) 2 'M") 

U 4 (Ui — Dj f j- 1>j ("Uj — 1?i )— ( ^ — {ViVj ) J v i 

———=^— v 2 v 2 — (y'.Vj^'Z — — — — — 

+ («j«j) 2 ), 

>(^-2^? + (^.) 2 )^| f 

where the last line follows since E\ 1 (vj) > implies i)J < ~. Combining ((78]) and (]83[) implies, 

£a lB («i) - Ei ln (vi(+)) > n(v 2 j - v[v,) 2 . (84) 

Together {76), (gU and d84j) imply (J74]). ■ 

Lemma 7.11: (DA Non-Decreasing Magnitude for any Communication Path) Any communication path C?j /y\ tl 
C[o,ti] implies, 

«i(*ife')(+)) 2 > Vi(*ofe')) 2 • (85) 
Proof: The first line in (|62| implies that for any signal S 1 ^ (tg J , t l ( ') , 

0<(t?(+)) 2 >«i(#') 2 ■ 
Thus for any communication path C[t (ij) H(ij)] defined as in ((33]) . 

6 i (*?* W) (+)) a >^ W )(*?* W) ) a 

> W*? W)Wl (+)) 2 

>^ W) - 1 (*^ W5<fc(MJ " 1 ) a 

> Wi^'^'W^- 
where every second inequality holds due to Lemma 17.71 since 9 > 9-1 for each q = 1, 2, . . . , fe(ij), where 
£ = j and 4(y)+i = *■ We then obtain ((85]) again by Lemma [777] since 0j 

%(io 15 ) 2 ^ %(*o(«i)) 2 because ti(ij) > t 1 * Cy) and i (ij) < respectively. ■ 
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Lemma 7.12: (Error Expression for Cm tl \ satisfying ([35]) ) For any communication sequence CV tl i satisfying 
(|35j the total reduction in normal consensus squared error E 2 (C[o ! t 1 ]) defined in (|42l) is, 

E 2 (q , tl] ) = EIU ( E liM°)) ~ El 1 Mh(+)))) 

= ^-Eix^| ln («i(*i(+)) (86) 

= E^eN^C^,*!]) > c K.t{] € svsc 

— n 

Proof: The first line follows from (A7),(A8) and the definitions (|4T|) — (|43|) . The second line in (|86|) is due to 
the initialization (|45 1) . The third line in (|86|) follows since any communication sequence C^o.ti] satisfying (|35j) can 
be partitioned into an infinite number of disjoint sequences C^ t i >t «] € SVSC. The last line in (|86|) follows since the 
minimum error of any normal consensus estimate is 0. ■ 
Lemma 7.13: (Vanishing Reduction in Error for Cm tl i satisfying (|35jl ) For any communication sequence Cm ^1 
satisfying (|35j) there exists an integer £ e such that, 



E'(C M )<s , V£>£ e , (87) 

for any e > 0, where E 2 (C[ t < |t <]) is defined by (j42]l . 

Proof: The third line of (1861) implies that for any CVq satisfying (|35l) the quantity E 2 (C[o ! t 1 j) is the sum of 
an infinite number of non-negative terms E 2 (C[ t * t «-j), the fourth line in (|86|) implies E 2 (C[ . tl ]) is bounded above, 
thus (|87|) follows by the monotonic sequence theorem. ■ 
Lemma 7.14: (DA Lower Bound on Reduction in Error for any C[ tl ] satisfying (|34l) ) Any communication 
sequence Cu 0ttl ] satisfying (|34| implies, 

E 2 (q Wl] ) > n(min 4ev {w t (ii(+)) 2 } 

- max ie y!XM 2 }) > ( 88 ) 

> . 

Proof: The first line in (|88l) holds for any communication sequence C[ to . tl ], 

E 2 (q f0 , tl] ) = E7=i (^l lB («<(*o)) - ^i ln («i(ti(+)))) 

=Er =1 (^(^(+)) 2 -^(M 2 ) 

> n( min ieV {'Si(ii(+)) 2 } - max ie v{wi(io) 2 } ) • 
To prove the second line in (1881) it is required that C[ t0itl ] satisfies (l34l) . We define, 

£ = arg, 4 min. ie v{^(ii(+)) 2 } , 
Z = arg 4 max ie v{^(io) 2 } • 

Since C[ t0:tl ] satisfies (|34[) there exists a communication path C~^,^ t <E C[ i0)tl ]. The Lemma r7.11| then 
implies, 

%(ti(#)(+)) 2 >%(i (#)) 2 • 
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The second line in (|88|) then follows because, 

mm tev {v l {t 1 {+)) 2 } = Mh(+)) 2 

> ij(to(M)) 2 

> vj{to) 2 = max ieV {^(to) 2 } , 

where the first and third inequality hold due to Lemma 17771 because t\ > t\{U) and to(££) > to respectively. ■ 
Lemma 7.15: (DA Local Convergence of Normal Consensus Estimates) For any communication sequence C[o_ tl ] 
satisfying (|35j) there exists an integer £ x such that for all I > i x , 

(«<(#') - %(4 J ')) 2 < X ■ V S«(#",tf) G C [t , )tf] , (89) 

for any % > 0. 

Proof: For any communication sequence Cm^i satisfying (|35]) the Lemma |7 . 1 31 implies there exists an integer 
£ £ such that (|87|) holds for any e > 0. For all I > l e we thus have for any signal S 1 - 5 '(t 3 ,t±) G t *j, 

«i(*i i ) 2 -«i(*o i ) 2 <*i(*i(+)) 2 -«j(*S) 2 , 

<E; = i(^(ti(+)) 2 -«r(^) 2 ) , (9°) 

= E 2 (q t , it ,,)<e. 

Note that the first inequality in (|90|) holds by Lemma 17771 since > £q and < t{ for any (t J , t^) <E C[ t i t *j. 
The second inequality in (|90)) holds since, 

E"=l («r(*f(+)) 2 -fir(*S) 2 ) 

= («^(+)) 2 - %(4) 2 ) + (%(ti(+)) 2 - ^) 2 ) (9D 

where 

E (*r(*?(+)) 2 -«r(*S) a ) >0 (92) 
reV\{ij} 

holds due to Lemma [7771 and, 

vM(+)) 2 - Mt'o) 2 > mm rev {v r (t[(+)) 2 } 

- max reV {^r(^o) 2 } (93) 

>0, 

where the second inequality in (|93j) holds due to Lemma 17.141 since C^i t «j G SVSC and thus satisfies (f34|) . 
Together (|9T|) . (|92| and (j93[) imply the second inequality in (|90"1) . Applying Lemma 17.101 to Lemma |7 . 1 3 1 implies 
that for all S ij (ti J ,t^) G ^ and £ > 4, 

e > E 2 (C [t , itf] ) 

>E 2 (S" (#',#')) , 

= ^| ln (« i (4 J '))-^| ln («i(*i i (+))) - (94) 
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for any e > 0. For notational convenience denote Vi = Viifii) and vj = Vj^). Combining (|90|) and (|94)) implies 
that for any e > there exist an integer £ e such that, 



v'j(vj - Vi) < \fe/n , 
for any S lJ (t^ , t\ j ) e C [t j and £ > £ £ . To obtain J89| note that d95j implies, 

{vi-Vj) 2 = v 2 - 2v'^j + tf$ , 

= v 2 -z) 2 + 2^(^ -«<) , 



v 2 -v 2 < e , 

(95) 



(96) 

< e + 2-^/e/n , 

< Ve(l + 2/^) , V ee (0,1]. 



We thus define e(x), 



?w = (tt§^) ■ (97) 

For any x € (0, 1) and e e (0,e(x)] the result J89| then follows from (|96|, ■ 
Lemma 7.16: (DA Properties of the Normal Consensus Update) Upon reception of any signal S y (t^ , 1 1 ( ) the 
normal consensus estimate t>i(^(+)) that results from the update problem (|53p will satisfy, 

Oi^y^iC*?) -*<(*?'(+))) <° • 08) 

Proof: Let us define ti, 

1 2 

u> = arg^ min (5 l n ) , (99) 

« e span{i) 1 (^(+)),w l (t l /),ej n 

where Vi(tY {+)) is given by (1531) . Note that € span{ui(^), Vj(ig J ), ei} implies, 

span{vi(t\ J {+)),Vi(t\ 3 ) 7 ei} C span-jv^ 4 /), -Oj(^), ej . (100) 

From (|100[) we have, 

E± ln (w) >E j . ln (v i (t^(+))) , (101) 
and thus combining (|101[) and (|60)l implies, 

* 2 <^(^'(+)) 2 . (102) 
Next observe that since w is defined by (l99|) , if (| 1 02[1 is applied to the signal S n (t l f ,t± (+)) then, 

E 2 (S^" ; ^(+))) ^El ln {v^{+)))-El ln {w) , 
= w 2 -{; 4 (^(+)) 2 <0 . 

Applying Lemma 17.101 to the signal S u (fiy , tj 3 (+)) then implies, 

>E 2 (5»(4 j ,4 J '(+))) 

>max{^(^') 2 -« 4 (^'(+)) 2 , (104) 

nfa^Wtf) -*<(*?(+))))*} 
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where the first line follows from (|103|) and the last line implies (|98l) . ■ 
Lemma 7.17: (DA Vanishing Change in Normal Consensus Update) For any communication sequence Cm^-A 
satisfying (1351) there exists an integer £ e such that for all I > £ s , 

(««(#'(+)) - Vi(t\ j )) 2 < e , V € tf M , (105) 

for any e > 0. 

Proof: Recall that Lemma [7.131 implies that for any communication sequence CV^i satisfying (|35j) there exists 
an integer £ £ such that (|87j) holds for any e > 0, we thus observe for £ > £ £ , 

e >&{C KA] ) 

> e 2 (s«(*W)) , v e C [tg)tf] (106) 

= *i(*?"(+)) 2 -« j (t?') 2 . 
For all £ > £ £ and signals S ij (t^ ,ii) <E C|- t « ( «] we then have, 

(«,(#(+)) - mi)) 2 = «i(4 J '(+)) 2 - W) 2 

+2« 4 (4 j )'(« i (ti i ) -**(*?(+))) 

<E 2 (S«(t^))< £ , 

where the first inequality follows from Lemma 17.161 and the second inequality from Lemma 17.171 ■ 
Lemma 7.18: (DA Vanishing Change between Normal Consensus Update and Signal) For any communication 
sequence CW^ satisfying (|35|) there exists an integer £ 1 such that, 

(«,(#'(+)) - %(^')) 2 < 7 , v e C[^], (107) 

for all £ > £ 1 and any 7 > 0. 

Proof: For any communication sequence CWj] satisfying (|35|) the Lemma 17.151 implies that (|89|) holds for 
any x € (0, 1) and e G (0,e(x)], where e(%) is given by (|97]l . The Lemma |7. 171 implies that (1 1 05[) holds for any 
e > 0, thus for any £ > £ £ ( x ) and S^it l Q ,t % f) € (7j t « t *j the triangle inequality then implies, 

y/(viW(+)) - vtf>)) 2 < yf (*$(+)) - v^)) 2 

< VHx) + Vx, 

<2^X, VxG(0,l). 

Any x € (0,7/4] and e G (0,e(x)] thus yields (|107j) for any 7 e (0,4]. ■ 
Lemma 7.19: (DA Network Convergence to Average-Consensus) For any communication sequence Cm^] satis- 
fying (|35j) there exists an integer ^ such that for all £ > 

2 (*(*?(+)) - -l n ) 2 < £ , V C7 [tSili] g C [0ltl]l (108) 

1—1 

for any £ > 0. 
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Proof: Since Cp ;tl ] satisfies ((35]) we have C^ t i t «j <E SVSC for each f eN, thus there exists a communication 
path Cft (^-) tl /jj)i G C[t*.tf] for an y • 6 V, j 6 V-i, and I £ N. For any i e V, j £ V_ 4 and £ £ N the triangle 
inequality then implies, 

v /(^(^ 1 (+))-^(^)) 2 

< Ey» ( tS» ' ,«?)ecf4 M)|tlM)1 

V(M*?(+))-*«(*S p )) 2 (109) 



where we define C* [t . ^ = \ 



(ij),tl(ij)] 



,wj — 1° v'-o ><-i ; fc °[tg,tf] • 

for g = 1, . . ., fc(ij), where 4 = j, 4 fo)+ i = i, (110) 

Ql m+l {ij) = {S im {tt n ,tT) eC M : 
4 m >*i(ii)} ■ 

We clarify that the RHS of (1109 [) includes the differences between the received normal consensus vector Si , ifo 9 ~ x ) 
and the updated normal consensus vector (i 1 ?£ ' !_1 (+)) that results from each signal contained in the communica- 
tion path CSjto) txiij)} ^ ^[to tf]' E ac h set Q e q (ij) defined in (| 1 1 0() contains the signals received at each node after 
the respective signal in communication path was received, but before the respective signal in the communication 
path was sent, as is required for an application of the triangle inequality. The set Qj.uj\ +1 (ij) contains the signals 
received at node i after the last signal in the communication path C^u^ tiUj)] was rece i ve d, but before or at the 
end of the communication sequence C^t t ij, again this is required for application of the triangle inequality. 

For any communication sequence Cm )tl i satisfying ([35]) the Lemma |7 . 1 7 1 implies there exists an integer £ e such 
that (|105[) holds for any e > 0. Likewise, for any communication sequence C[o^ tl ] satisfying (|35|) the Lemma 17.181 
implies there exists an integer £ 7 such that (|107[) holds for any 7 > 0. Thus for any 7 £ (0, 4] if we let \ € (0; 7/4] 
and e £ (0,e(x)] then for any £ > i^M, 



v /(%(4(+))-^(io 1J )) 2 

+Eg )+1 l^(y)IV^W: ( m ) 

< In/7 , 

where the second inequality holds since e(x) < 7, and the last inequality holds since every signal contained in 
C[ t « t ij is represented by at most one term on the RHS of (1109 [) . that is, 

k(ij) + 1 
q=l 
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Due to any £ > and 7 G (0, £/(n|C [t « -t *] |) 2 ] then implies, 

(%(*!(+)) - M*^)) 2 - (%(**(+)) - . (112) 

for any G V 2 , where the first equality holds due to Lemma [7791 The result (| 1 08 1) then follows from (|1 12|) 

since 

EIU N*f(+))-^) 2 

= EIL 1 £"=i (%(*!(+)) -««(*o IJ ')) 2 , 

< n 2 (£/n 2 ) , 

where the first equality holds again due to Lemma |7?9l ■ 
Proof. (Theorem |4j7]) Lemma |7 . 20117 . 22l Overview of Proof. The Lemma 17.201 proves the OH normal consensus 
update (|22[) . As stated in Lemma 17.21 1 the update (j22| implies the error of each element of every normal consensus 
vector is a non-increasing function of time, and thus (j22| will imply the conditions stated in Lemma 17.221 are 
sufficient and necessary for any node to obtain average-consensus. The Theorem 14 . 7 1 then follows immediately from 
Lemma 17.221 

Lemma 7.20: (OH Normal Consensus Estimate Update) Applying (|20[) and (j2Tjl to the optimization problem (|9| 
yields the OH normal consensus estimate update t>i(t l i (+)) defined in (1221) . 

Proof: If Vj(t%) = il„ then = (t% t* j in this case (gOj together with ® implies ((48]). The update 
(|9|) can thus be re-written as (|49|) . Since Sjit^) = s(0) is known we can let Si(t^(+)) = s(0) and thus obtain 
^i(^i' J (+)) = n^-n as the unique solution to (|49p . Note that this coincides with (|22)) . Next assume that Vjit]^) 7^ —l n 
and thus ^ S[ J (t' J ,t\ J ). Under (J20j) and ^2TJ we can re-write © as, 

Vi(ti(+))= ar Sw min ( -5 ~^ 1 ™) 2 ' 

s.t. ® holds, given s^tf) = Svi(t\ j ), (113) 

Sj(0) = Sej, Si(0) = Sei. 

Given that Sj(tj') = Sui(i^), Sj(0) = Sey, and Sj(0) = Se^ are known, the set of vectors Vi(i^ (+)) for which ((8]) 
holds is spanjw^t 1 /), ej, e^}, thus (| 1 1 3 1) can be re-written as, 

1 2 

= argc; min («--l„). (114) 

v G span{ui(^), ej,ei} n 

If t5i (tf' ) G span{ej,ei} then (|1 141) becomes, 

«<(#(+)) = ar g{i min- g span{e . )e . } (5 - ±ln) 2 , 
= V(OHi)V (:oH1) -ln , V(ohi) = [ej,e%] ■ 
Since e; is linearly independent of ej we then have, 

«*(*?'(+)) = V { oHi)(V( QH1) V (OB1) )- l V( qH1) ^l n 

= ^V (0ffl) (7 2 )- 1 l 2 , (115) 
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Note that if ii(f{ > ) £ spanje.,, e^} then the last line in (|1 1 5[) coincides with (|22l) . Next assume Vi(tY) ^ spanjej, e 2 }. 
In this case (|1 14[) can be expressed, 

Vi(t[ j (+)) = V i oH)V ( + OH) ±l n , 

= v (OH)(V ( ' 0ff) V(oH)) _1 V ( ' 0ff) il„ 

Recall the discrete set of vectors M™ j_ is defined in (|25l) . Note that the initialization (|45l) implies -0^(0) = i and 
thus «j(0) € KJi. Also note that (flB)) implies Uii(4 j (+)) = ^ and € K™ 1. We thus assume, 

) = i , QiffleM*! ■ (116) 
n '» 

Observe that under (jl 16P if the result (|22|) is proven, then the assumptions (|1 are valid. For notational convenience 
denote Vi = Vi(t % ^). Given (|116[) the matrix (V7 OJ3 -\ V(0-ET) ) has an inverse (|117|) below, 

U-l„ 1 Uij \ 

1 1 



\ %■ 1 



/ 



l V ('c«r) V C°H)l 



-1 {v'jln-Vi 



\ 



where the determinant | Vf OH ^ V(oh) I can be computed as, 



\V/o H) V ( oH)\ = n- 6 (v! i l n -l-v ij ) 



(117) 



Next observe that, 



v \ 1 



1 

V 1 J 



(118) 



From (|118p we observe that right-multiplying (1117[) by ^('ojy)^ln an( l left-multiplying by V(oh) m en yields ((22]). 



V(OH)(V( OH) V(OH)) 1V (OH)n ln 

( 



e'^-x-a^ v (oh) 



U t -l n - 1 - «ij 



y w-l„(l - %) + Ujj - 1 j 



Lemma 7.21: (OH Element-Wise Non-Increasing Error of the Normal Consensus Estimate) The error (vu(t) 
1/nj 2 of each element vu(t) is a non-increasing function of t > for all (i,£) £ V 2 . 
Proof: The result follows immediately from the normal consensus update (|22|) . 
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Lemma 7.22: (OH Normal Consensus Estimate Convergence to Average-Consensus) Under the OH algorithm, 
any node i G V obtains average-consensus by time t\ for some communication sequence C\$ >tl \ iff at least one of 
the following two conditions holds, 

• (Cli): there is a signal (t$ , t % ( ) G C[o.ti) f° r eacn J <= V-i- 

• (C2i): there is a communication path C^ o ^ tl ( i ^] G C[o.ti) from at least one node I such that S io (tfj 1 ,t^) G 
C[o,t («)) for all j e V_*. 

Proof: (Sufficiency.) If a communication sequence CWf-o implies (Cli) for some node i then under the update 
(|22p there will exist a time t l f G [0, ii ) such that Vij(t^(+)) — — for each j € V_i, and thus Lemma P7.21I 
together with (|45|) imply £>j(ii) = — l n and hence by ([8|) the node i reaches average-consensus by time t\. If 
a communication sequence CmM implies (C2i) then by the previous reasoning the node I will obtain average- 
consensus by time to(i£), and thus by the update ([22"]) any node j G V that £ sends a signal S^{ipQ ,t{) G C 1 ^^) tl ) 
will obtain average-consensus at t{ e (+). If the node £ has a communication path Our^ tl M\\ G C[o,ti) to node i 
it then follows that node i will have obtained average-consensus by time ti(i£)(+), and thus by Lemma 17.21 1 node 
i obtains average-consensus at time t\. 

(Necessity.) Under the OH update (|22|) . if there is not a signal S 1 ^ (t^ ,t^) G C[o,ti) tnen there will not exist a time 
t G [0, ti) such that iij(t) = i unless there exists a communication path Ctf,^ tl M\] G C'jo.t!) from some node ^ 
such that S*i(tQ ,ti) G C[o i t (^)) for all j G V_£. It thus follows that node i cannot obtain average-consensus by 
time ti for any communication sequence Cm,*!) tnat does not imply either (Cli) or (C2i). ■ 

Theorem \4.7\ OH Network Convergence to Average-Consensus) 

Proof: The condition (C) stated in Theorem 14.71 is equivalent to when either (Cli) or (C2i) hold for each node 
i G V, the result thus follows immediately from Lemma |7. 221 ■ 

ProoffTheorem |4~9l ) Lemma l7.23H7.35l Overview of Proof. Similar to Theorem 14.51 we show that every normal 
consensus estimate Vi(t) satisfies the normalization property (|54]), the "zero local error" property (|64"1) . and further- 
more the discretization Vi(t) G j_. The Lemma 17.281 and Lemma 17.291 shows that the update ()123|) respectively 
implies the error of each normal consensus estimate is non-decreasing with time, and that the normal consensus 
estimate will not change unless there is a reduction in error. We then show, analogous to Lemma 17.131 that the 
reduction in error that results from any signal will eventually vanish if Ciq tl i satisfies ([37]) . Due to the discretization 
i>i (t) G Rq j_ this implies the reduction in error that results from any signal will eventually strictly equal zero if 
C[o,t!] satisfies (f37|) . see Lemma 17.351 When this occurs we can show that ii(t) = —l n by utilizing Lemma 17.301 
Lemma 17.311 together with the "zero local error" property (|64|. hence (|44|) holds and so by Definition 17. 1 l a network 
average-consensus is obtained. 

Lemma 7.23: (DDA Normal Consensus Estimate Discretization) Every normal consensus estimate Vi(t) satisfies 
Vi(t) G K™ i for all i e V and t > 0. 

Proof: Note that the initialization (|45|) implies Uj (0) G K" j_ . The optimization problem (|26|) requires that any 
solution Vi(t(+)) satisfies Vi(t(+)) G x- Under the DDA algorithm, the assumption (A7) implies every normal 
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consensus estimate remains fixed unless updated via (|26l) , it thus follows that Vi(t) E K™ A for all i £ V and t > 0. 



Lemma 7.24: (DDA Consensus Estimate Normalization) Every normal consensus estimate Vi(t) satisfies (l54l) , 
and furthermore, 

(V(i)) 2 - Jv'W'Vi , V i e V , V t > . (119) 

Proof: The Lemma 17.231 implies, 

^iW'ln = <>«(*)> 

= 7T E<?eV : C«(t)=i (n) ' 

= E.ev : (£) , d 2 0) 

= ELi fi «(*) 2 . 
= «i(t) 2 • 

If ia{t) elji then G MgT 1 , thus a similar argument to (fT20|) implies (fTT9j) . ■ 

Lemma 7.25: (DDA Local Zero Error Property) Every normal consensus estimate fii(t) satisfies (|64| . 

Proof: The Lemma l7T2l implies 0^(0) = ^ for each i£ V. Next observe that under (A7) the estimate v%(t) will 
not change unless a signal S* 3 (t^ ,t^) is received at node i. If a signal is received then r)j(t) is updated by (|123[) 
given below. We now show that under the assumption (|64|) . the solution Vi(t^ (+)) specified by (|123|) will imply (|64|) 
for every set vectors {e*, If V^^'V^o 7 ) = then fii(4 i (+)) = «i(4 J ) + ~ M*o% 

and thus, 

M*i (+)) = + M*o ) - «ii(*o J ) » 

_ 1_ 

n 

If vr(t\ 3 YvJ l (t^) > and £p(^) 2 < vr% j ) 2 then (+)) = + ^(I - and thus, 

*<(#(+)) = «i<(*o) + S - %i(*o) > 

n 

Finally, if 1)? ($ ') > and v~ l (t\ r ) 2 > vj\tf) 2 then = and thus d64| follows by 

assumption. ■ 
Lemma 7.26: (DDA Normal Consensus Estimate Magnitude Equivalence to Error) For any two normal consensus 
estimates «,-(<) and Vj(t), 

Vi(t) 2 >Vj(t) 2 £l lB («<(*)) <^a ln («i(t)) • (121) 

Proof: For any normal consensus estimate Uj(t), 

^I ln («i(*)) =(«i(*)-^n) 2 , 

= « 4 (t) 2 -2i« 4 (t)'l n + I , (122) 
= ^-«i(*) a 

where the third line holds due to Lemma 17.241 The equivalence (|121j) then follows directly from (|122|) . ■ 
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Lemma 7.27: (DDA Normal Consensus Estimate Update) Applying (|27|) and (|28[) to the optimization problem 
26j) implies the DDA normal consensus estimate update Vi(t % f (+)) can be defined as in (|29|) — (|30| . 
Proq/: Note that |29} - (J30j) implies, 

s<(*?) + %(4V , if V(4 j )V(*o) = o, 

'-fi't+n = ^ %(4 j ')+ei(i-^(4 j ')) , if vw^v^o ) > o . «rw < *r(#') 2 > < l23 > 

, if ar^'JVC^) > o , orW > fi7'(*o ) 2 ■ 

We thus will prove that ([27 P and p8]) imply optimization problem (|26[) yields the DDA normal consensus estimate 
update it(t\ 3 (+)) defined by (fT23|) . Under |27| and (J28j> we can re-write |26| as, 

(+)) = arg^ min (y - ^lnf , 

s.t. J8) holds, « G R£ i, given §i(t[ j ) = Svi(t\ j ), (124) 

Given that Si(tY) = Svi(t*i), = S%(to J ), and Si(0) = Sei are known, the set of vectors Viifil (+)) for 

which ([8]) holds is span-ft^t^ 7 ), Vjiffi), e,}, thus (j 1 24[) can be re-written as 

WW) = a ^ min , espan{ , )i ( t «) >% (4i), ei} nR« 1 (^-^») 2 ' 

Q,_ ^ (125) 

= arg B = (oT) i (t?) + w J -(t§ ? ) + c i ei )nR* ) i min(Q - b - c) • 

For notational convenience denote = Uj(t^ J ), fL = Vj^), and (+)) = Note that the constraint in 

(11251) can be expressed as follows, 

av u + bVjt + c-e ie G (0, -) , V I G V. (126) 
n n 

Due to Lemma 17.251 the i t/l constraint in (|126[) is, 

a l + Ma + c ^ (°. £) ' =* 
—a + bvjin , 
1 — a — M)jjn . 

If c = —a + bvjin we then have the candidate update v\ 1 \+), 

(+) = aVi + bvj + ^ei(-a + bvjin) , 

^ (1) ( + ) 2 4f(+)'ln 



where the magnitude (+) 2 is computed using Lemma 17.241 If c = 1 — a + bvjin we then have the candidate 
update v\ (+), 

w,p ) (+) = ow, + 6uj + £e,(l - a + frO^-n) , 

^ (2) (+) 2 =^ (1) (+)'l. (12?) 
= avf + 4- — — bvji/n , 
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Since ea = for all I ^ i, the i th constraint of (| 1 261 is the only constraint involving the optimization variable c, 
thus by applying Lemma 17.261 it follows from (|127l) that if c ^ 1 — a + bijin then the resulting solution i>i(+) 
cannot be a solution to (|125l) . hence c = 1 — a + bv^n. Next observe that if vu — Vjt = then the £ th constraint in 
()126l) places no restriction on a or b, thus due to Lemma 17.231 and Lemma 17.251 we can consider the three possible 
scenarios posed by (|126|) given the i th constraint is satisfied by (jT27j) : one constraint, 

(MIA) a (0) + 6(I)e(0,i) => be (0,1), 
(M1B) o(I) + 6(1) G (0,1) => 6 6(1-0,-0), 



two constraints, 



or three constraints 



(M2A) a (0) + 6(i)6(Q,i) 
«(£) + &(0)€(0,A) 
(M2B) a (0)+6(i)6(0,i) 

(M2C) a (±) + 6(0)e(0,±) 
^) + ^)6(0,i) 

(M3) a(0)+6(i)e(0,i) = 
a(I) + 6(0) 6 (0,i) = 



6 6 (0,1) 

• ae (0,1) , 

• fee (0,1) 

> a 6 (1 - 6,-6) , 
ae (0,1) 

> b G (1 — a, —a) , 

6e (0,1) 

a € (0,1) 
6 € (1 — a, —a) . 



If vj 



= and there is one constraint, then Lemma [7.251 implies v i 1 = and hence (MIA). In this case we 



have Vi = — e, and the following candidate solutions, 

+ ^C 1 ~ a ) = ^> = 

fl n e » + % + n 6 ^ 1 ~~ a ~ VjiU), (b = 1) 



Vi(+) 



(128) 



where the second line in (|128|) simplifies to £ij 
magnitudes, 

^(+) 2 = 



n 1 \ 



VjiU 



). The solutions in (|128p possess the following 



(6 = 0) 
(6 = 1) 



and thus since Lemma 17.251 implies v^ 1 2 > 0, the optimal solution is, 

Vi(+) = Vj + ie,-(l - Vjin) , 

— ^2 Vj , 

where the last line follows under the given assumption = ^e^. If v~ % 
then (M2A) necessarily follows. In this case v~ % 2 > and we have the candidate solutions, 



Vi( + ) 



(129) 



= and there are two constraints 



n L ' 


(a 


= 0, b 


= 0) 




(a 


= 1, 6 


= 0) 


V 3 + n e i(l _ > 


(a 


= 0, 6 


= 1) 


Vi -j- Vj i 


(a 


= 1, b 


= 1) 



(130) 
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The solutions in (|130|) possess the following magnitudes, 



1 

7? ' 




(a 


= 0, 6 = 


0) 


«i i2 " 


i 

~ ' 


(a 


= 1, b = 


0) 


Of 2 - 


i 


(a 


= 0, 6 = 


1) 


vV 2 - 


,*.-« 2 , 1 


(a 


= 1, 6 = 


1) 



and thus since Lemma 17.251 implies v J : 4 2 > 0, the optimal solution is, 

«*(+) = «j + Uj - ei%j . 



(131) 



If v i 1 'dj 1 > and there is one constraint then we necessarily have v i 1 = « ■ 1 and thus (Af IB). In this case the 
candidate solutions are, 

avi + (1 - a)?),- + iej(l - a - (1 - a)u,-,-n) 
a«i — aij + ~ej(l — a + avjin) . 
Note that the first and second line in (|132[) correspond respectively to when 6 = 1 — a and b = —a, also note that 
each line simplifies respectively to + h,Si(l — Vjiii) and Aej. The solutions in (|132l) thus possess the following 
magnitudes, 

2 ; V l2 + ^' (6=l-a) 
£ , (6 = -a) , 

and since Lemma 17.251 implies {f 2 > 0, the optimal solution is, 



Vi(+) = Vj + ~e,-(l - Vjiti) 



(133) 



Vi(+) = < 



^(+) 2 = { 



-I 2 



(134) 



where the last line follows under the assumption 4 = v, \ If 4 'w^. 4 > and there are two constraints, then if 
v~ l 2 < if 2 it follows that (M2B) holds. The candidate solutions in this case are, 

± ei , (a = 0, b = 0) 

Ui , (a = 1, 6 = 0) 

-it + Vj - ie,(2 - fij-jn) , (a = -1, b = 1) 
% + iei(l - UjiTi) , (a = 0, 6 = 1). 

The solutions in (| 1 34[) possess the following magnitudes, 

£ , (a = 0, 6 = 0) 

(a = 1, 6 = 0) 
2 + ^ , (o = -l, 6 = 1) 
, „=. , (o = 0, 6 = 1). 

The assumption uf 2 > Of 2 then implies the optimal solution, 

1 



Vi(+) = vj e»(l - %n) 



(135) 
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If v i 1 1 > and there are two constraints, then if v i 1 > 1 it follows that (M2C) holds. The candidate 
solutions in this case are, 



©i(+) 



ie, , (a = 0, b = 0) 

v j + ^ e *( 1 _ ' ( a = °> b=1 ) 



- ©j + -a , 



The solutions in (|136|) possess the following magnitudes, 



^(+) 2 

l 



1 




n 2 






2 












2 


V 










2 


V 





(a = 1, 6= -1) 
(a = 1, 6 = 0). 

(a = 0, 6 = 0) 
(a = 0, 6 = 1) 
, (a = 1, 6 = -1) 
(a = 1, 6 = 0). 



(136) 



The assumption v ■ 1 2 < £>j 2 2 then implies the global solution, 



u»(+) = w< 



(137) 



(138) 



If v i 1 r Vj 1 > and there are three constraints then (M3) necessarily follows. The candidate solutions in this case 
are, 

- ei , ( a = 0, 6 = 0) 

Ui(+) = ^ f)j + iei(l - v^n) , (a = 0, 6=1) 
©i j (a = 1, 6 = 0). 

The solutions in (|138|) possess the following magnitudes, 

^ , (a = 0, 6 = 0) 

<'. < -- > 2 = <! 2 + ^ , (o = 0, 6 = 1) 
i , (o=l, 6 = 0). 



The assumption t3 ■ 



In contrast, the assumption u ■ 



then implies the optimal solution, 

Vi(+) = Vj + -eAl - vuri) 
n 

' 2 < 2 implies the global solution, 



(139) 



(140) 



Combining ([1291), (USD, p3]) . (TT351) . (fl37j) . ([1391) . and <jI30|) . together imply (JT23J. Note that if vj 1 2 = fi" 1 2 
and vj 1 ^ v~^ % then there are two global solutions to (|125p . By specifying (| 1 37[) and f| 140[) we have chosen the 



global solutions that are necessary for Lemma 17.291 which is in turn necessary for the DDA algorithm to obtain 
average-consensus under the sufficient communication condition stated in Theorem 14.91 ■ 
Lemma 7.28: (DDA Normal Consensus Estimate Non-Decreasing Error) The error of every normal consensus 
estimate Vi(t) is a non-decreasing function of t > for all i G V. 
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Proof: The Lemma |7. 271 implies that upon reception of any signal (t^ the normal consensus estimate 
Viiff) is updated to a solution of (|125p . thus 

Vi(t[ j (+)) e span{^(^'),%(^),eJnR™j_ . (141) 

The Lemma 17.231 implies ii(tY) & Kg j_, thus (|141|) implies that any candidate solution (+)) that does not 

satisfy Ei ln '(+))) < £i ln (v(4 J )) cannot be a solution (fT25|) . ■ 

Lemma 7.29: (DDA Fixed Normal Consensus Estimate Given No Reduction in Error) Under the DDA algorithm, 
for any signal S*-* (tg 7 , t % l ) we have, 

El(vi(t[ j (+))) = => «i(4 J '(+)) = d42) 

Proof: Applying Lemma 17.261 to the LHS of (|142[) implies, 



^ (*(*?'(+))) =^ (*(*?")) 

To prove (I142P we thus have only to show, 



«. K(4 3 '(+))) 2 = (^(*i j )) 2 



*(#'(+)) ^ fii(*?') (*i(^(+))) 2 ^ (^(*l j )) 2 • d43) 
Under the update (|123D , to prove (| 143[) we need only to show that either of the two cases, 



if Or i (t«)'t>r*(t«)=0, 



(144) 



fii(*?'(+))=fl i (t§'')+Ci(^ . 

if v(4 j ') v(*o) > °- v(^) 2 < er(*o') a 

will imply uf (4 i (+)) > uf If ^(tl^'vp^) = then ([T44]l together with (|54} imply, 

«?(4 J '(+)) = £*<(*?(+))'!« , 

= 71 (^(4 J ) + "j(*0 ) ~ %i(<0 ) e i) , 
= i(«i(4 j )'ln + «i(*o J )'l« ~ %(#)) . 
= «?(*?) + H r< (*o) , ln-l. 
>«?(*?'). 

where the last inequality holds because (J64j) implies vj l {t)'l n > \ for all j e V_; and t > 0. If i>~ 1 (tf )' vj l (i„ ) > 
and up(*i J ') 2 < vpitf) 2 , then due to (fT44l) it follows that, 

*?(*?(+)) =£«<(*?(+))%., 

= ^(*j(*o'')+ei(s-«i<(*o''))) / ln , 

= H%(4 i )'ln + i-% i (4 J ')) . 
= ^(4^-1 + ^), 
> 1(^(4^1^ + 1), 
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where the final inequality holds under the given assumption that v^ l (t 1 ^) 2 < vj 1 ^) 2 , and the second last equality 
holds due to Lemma 17.251 ■ 
Lemma 7.30: (DDA Lower Bound on Increase in Normal Consensus Magnitude) Under the DDA algorithm, for 
any signal S 1 ^ (t l Q ,t^) we have, 

(*r'(tf(+))) 2 > ^{(v(?)) 2 , (V(4 J ')) 2 } ■ 

Proof: ttvpffiyvj*^) = then Vi(t\ j (+)) = v l (t\ 3 ) + v J {tl ) r )~v jl {tf)e l and thus applying (pT9|> implies, 

(«r'(*?'(+))) a =^r(*?(+))'i»-i . 

= i(V(4 J ')'ln-l+V(4 i )'ln-l) , 

>max{(V(^')) 2 , (V(*o)) 2 }- 
If ^(t?') VW) > and vr\t\ j r < vj\^f then «*(#'(+)) = + ~ and thus, 

>max{(fip(t?')) 2 , (V(*o )) 2 } 
where the last inequality holds by the assumption v^' l (tY) 2 < vj 1 ^) 2 . Finally, if (fil) 1 vj* (t^) > and 
V(t?) 2 > ^(to? then «<(*?(+)) = and thus 

(V(4 j (+))) 2 = iV(4 J '(+))'ln-l , 

> ma X {(« i - < (4 J ')) 2 . (V(4 J )) 2 } , 



where the last inequality holds by the assumption l (t^) 2 > iij l (t l Q) 2 . 

Lemma 7.31: (DDA Upper Bound on the Magnitude (#,"*(*)) 2 ) If «f(£) = uf (f') for all (t,f) e [t ,ti] then 

(VW) 2 < (V'(O) 2 , v j e v_, , v e Mi]. 

Proof: The result follows since Uj(£) elj, implies i>ij(t) G (0, i) for all jeV and t > 0, thus, 

<v 2 (t')-i 2 At') > 



(V'(i')) 



Lemma 7.32: (Error Expression for Cjo^j satisfying (|37|) ) For any communication sequence C[o ytl ] satisfying 
371) the total reduction in normal consensus error from t = to t = ii(+) is, 

E 2 (q , tl] ) = E?=i Hi n (^(o)) - ^i ln («i(*i(+)))) 

= ^-eL^i w (<Wi(+))) (145) 

= E^ e N E2 ( C [tg,tf]) > C K,t[] € SVCC 
^ n— 1 
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Proof: The proof is identical to Lemma ITTH with |(86|), (|35|) , and SVSC replaced by (fT45|) , (|37|) , and SVCC 
respectively. ■ 
Lemma 7.33: (DDA Vanishing Reduction in Error for Cjo.tj satisfying ([37]) ) For any communication sequence 
CVtil satisfying (|3~7|) there exists an integer £ e such that, 

V 2 {C KA] )<e , V^>4 , 

for any e > 0. 

Proof: The proof is identical to Lemma 17.131 with (l86l) and (|35|) replaced by (|145[) and (|37|) respectively. ■ 
Lemma 7.34: (DDA Lower Bound on Non-Zero Reduction in Error) For any communication sequence Cr t0jtl ] 
we have, 

E 2 (q t0 , tl] ) > o => E 2 (q t0 , tl] ) > -1 . (146) 

Proof: Applying Lemma [7. 261 to the LHS of (|146p implies there exists some signal S ZJ (t 1 ^ , tff) G C[ tgtl ] such 

that, 

«?(*?'(+))>«?(*?') ■ (147) 

We now show that (|147|) implies v 2 (t\ 3 (+)) > i'f(t\ 3 ) + the result p46l) then follows directly by Lemma 177281 
together with Lemma 17.261 The Lemma 17.231 implies Vi (t) e R™ j_ for all i e V and t > 0, thus under (|147|) it 
follows that, 

*?(#'(+)) = £*(*?(+))'!» , 

= ^|{£ : fi«(#'(+))}| , 

(148) 

= : v ie (ti j )}\ ■ 



From (|148|) it then follows, 



\{l : >\{i ■■ m(4 j )}\ =* 

\{i : ««(#'(+))}! -1 >|{^ : 0«(t?)}| ■ 



Under the constraint Uj(t) G R™ j_ the last inequality in (|149|) implies (*'/(+)) > uf (i?) + Tpr- ■ 
Lemma 7.35: (DDA Existence of a Time for Zero Reduction in Error) For any communication sequence Cm^i 
satisfying (|37|) there exists an integer £ i such that, 



E^(C [tS , t!] ) = , V*>*£ . (150) 

Proof: The result follows immediately by applying Lemma 17.281 and Lemma 17.341 to Lemma 17.331 ■ 
Theorem I4.9F DDA Network Convergence to Average-Consensus) 

Proof: For any communication sequence Cm^i satisfying (|37j) the Lemma [7.351 implies there exists an integer 
such that (|150p holds. We now show that the condition, 

E 2 (C KA] )=0 , C KA] G SVCC (151) 
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implies (|44j) at t = £g, and hence by Definition 17. 1 1 a network average-consensus is obtained at t . Applying Lemma 
17301 to ([Bill implies, 

(^(4 J '(+))) 2 ^(^W.VjeV^, 

, • ^ s2 (152) 

>(V(*o J )) , ViGV_ 5 

where the last inequality holds due to Lemma 17.311 Likewise, applying Lemma 17.301 and Lemma 17.311 to (|15ip 
implies, 

(«7 i (*i t (+))) a >(f(0) 2 ,v je v_ ; , (i53) 

>(*f VjeV_ ; . 
Note that applying Lemma 17.291 to (|151[) implies that (|152|) and (|153[) can be combined to obtain, 

(rW = (f'(4)) 2 .viev. i => 

- £ = «, 2 (*S) - , v j e v_ ? . 

The second line in (|154[) together with Lemma 17.251 implies, 

%-(4) = ~, VieV, 

and thus Wj(<o) = 7^ In- Since C[ t « ( r € SVCC, the Lemma 17.291 together with the update (|123[) implies = 
il n for each j E V_j, thus (|44|) holds and so by Definition 17. 11 average-consensus is obtained at time t l Q < t%, ■ 
Remark 7.36: We observe that if the SVCC condition is defined by the condition (C) stated in Theorem 14.71 
then using the definition ([37]) for the IVCC condition will not imply Theorem 14.91 In other words, using condition 
(C) to define an SVCC sequence will imply there exist examples of IVCC sequences for which the DDA algorithm 
will not obtain average-consensus. This is why we have defined SVCC only by (|36|) . which is actually a special 
case of the condition (C) stated in Theorem 14.71 Furthermore, the DDA algorithm normal consensus update (11231) 
is only a global solution to (|26*| , it is not a unique solution. Under (|20)) and pT]) . the alternative global solution to 
(1261) is, 

' Vi(t\ j )+vM j )-^>i , ifV'(*?')V(*S')=o, 

%(4 J ')+^(^-%i(4 J ')) . ifV(4 J ')V(*o')>o, v(4 J ') 2 <«r(*o) 2 . 
m j ) , if sr^Vfto ) > ' v 2 (^) 2 > *7 i (^) 2 , 



(155) 



, if VC^JVw) > o . V(*?) a = ) 5 

The above remark still holds even when the alternative global solution (|155[) is used to update the normal consensus 
estimate; however, by randomly switching between the updates ()123|) and (|155[) leads to the following conjecture. 

Conjecture 7.37: Let the SVCC condition be defined by the condition (C) stated in Theorem 14.71 Suppose upon 
reception of each signal the normal consensus estimate update is defined by (|123[) with probability p e (0,1) and 
defined by (|155p with probability 1 — p. Then (j6| holds at time t = ii(+) almost surely for any communication 
sequence C[ ^ tl ] satisfying (|37| . 

The significance of the above result is due to the fact that condition (C) in Theorem 14.71 is considerably weaker 
than (|3o*l) . If Conjecture 17.371 holds, then by defining the DDA algorithm using the above randomized protocol, and 
defining the SVCC condition by the condition (C) stated in Theorem 14.71 the Venn diagram in Figfl]will then be 
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completely accurate. We note there are alternative definitions of the SVCC condition and OH algorithm that will 
also lead to the same Venn diagram as Fig[T] However, we know of no weaker sufficient condition than that stated 
in Theorem 14.91 for convergence of the DDA algorithm (|27[) — (|32[) . and this sufficient condition is based entirely 
on the SVCC condition (|36l) . 

B. Comparison Algorithms: Gossip and ARIS 

In this section we define the two comparison algorithms, Gossip and ARIS, in terms of the class of distributed 
algorithms Q, (0). 

I) Comparison Algorithm 1 (Gossip): The Gossip algorithm proposed in [3] implies a signal specification and 
knowledge set update defined respectively as (|156() and (I157[) below. 





Gossip Algorithm: 




Signal Specification: 




(156) 








Knowledge Set Update: 


(157) 






Initialization: 


/Ci(0) = {si(0)} , §i(p) = Si (0) ,V!£V. 


(158) 



We note that under the Gossip algorithm the only communication conditions proven to ensure average-consensus 
require instantaneous and bi-directional communication, thus implying 

,ij ,ij ,ji ,ji 

i Q — tj — r — i x , 

for any signal S 13 (i 1 ^ , ) € C[o,til- Under the assumption (|159p , any CVti] satisfying the IVSC condition (f35|) 
will imply the Gossip algorithm obtains average-consensus at time t = ii(+), that is (|6| is satisfied at t = t±(+). 
We note that in some works (e.g. ifTTIl . (2]) the Gossip algorithm is referred to as "pairwise averaging". 

2) Comparison Algorithm 2 (ARIS): The adaptation of the randomized information spreading algorithm proposed 
in |fl9ll that we will consider can be defined by the signal specification (| 160|) and knowledge set update (|161[) below. 





ARIS Algorithm: 




Signal Specification: 


S i J(tltV) = lC j (ti j )\{s i (QUn} 


(160) 


Knowledge Set Update: 


W^(*i i (+)),t»i(*i , '(+)),*i(0),»,n} 


(161) 
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Ki(0) = {ki(0),8 


■ fo) W'fnl «/>-(n) s-(o) ? 77 \ 




Initialization: 


fci(O) = o , 


Si(0) 


= Sj(0)/n , u)i(O) = e 4 , 




(162) 




w i (0) e i 




W\ q {Q) ~ exp { a „(0)} , 






V^ = 


1,2,. 


.,d , 9= l,2,...,r . 





We clarify that the (lq) th element in the matrix W*(0) is an independent realization of a random variable from 
an exponential distribution with rate parameter su(0), this is why the elements of each initial consensus vector 
Si(0) are required to be positive valued under any version of the RIS algorithm. We next define the ARIS update 
for each term in the knowledge set Ki{t l f (+)), we will omit the time indices for convenience. 



ARIS Knowledge Set Update Procedure: 



w 1 = < 



1, 

0, 



5[wi +Wj], if kj 



h(+) 



5[ei + Wj], if kj > 
if kj *\ 

kj, if kj > h.w 1 ^ l n 

kj + 1, if kj > ki, w 1 = l n 

ki, if kj — ki,w ^ l n 

ki ~\~ 1 , if k j 

ki, 



U>i(+) 



i — ki, w — 1 r 

j 

if w 1 i {1„,0} 



if kj - 



ei, if w 1 = 1 
Wi, if w 1 = 0, 



n 



~ exp {s«(0)}, if h(+) > ki 
mm{Wi q ,Wi q }, if h(+) = k t = kj 
W} q , if k 3 < h 

1,2,..., d, g = 1,2, ...,r, 



exp {s«(0)}, if % > fcj 



Wi q (+) 



mm{W J eq , W} q }, if fc, = ki, w 1 ^ 1„ 
Wl q , if ^ < fc 4 

£ = l,2,...,d, 9 = 1,2,. ..,r, 

^ = 7 E^i min{W/ 9 (^'), W7,(t?)} , * = 1, 2, . . . , d, 
= r K=i min{<(4 J ') ; , t = l,2,...,d. 



(163) 



(164) 



(165) 
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if kj ki . 






c ■ 

% 1 


\f Jc ■ — ]c lii^ ^Jz 1 
3 — 1 ' / n 




Si(+) = < 


kiSi-\-w i 1 /n 


if fcj = fc l; Ul 1 = 1„ 

if fej > h, w 1 ^ 1„ 


(166) 


fei+1 ' 

Sj, 






if fcj > fcj, w 1 = l n 











The integer r is an RIS algorithm parameter that affects the convergence error of the algorithm; as r increases 
the algorithm is expected to converge closer to the true average-consensus vector s(0). Based on the strong law of 
large numbers we make the following conjecture. 

Conjecture 7.38: For any Cro^i satisfying the SVSC condition (|34| . both the RIS algorithm proposed in |[T9l 
and the ARIS algorithm (|160p — (|166|) imply ((6]) at time t = ti(+) almost surely in the limit as r approaches 
infinity. 

As explained in Sec lV-Al the total resource cost of the RIS algorithm increases on the order 0(rd), and likewise, 
the total resource cost of the ARIS algorithm increases on the order 0(rd + n), thus it is not practical to assume 
r can be made arbitrarily large as the Conjecture 17.381 requires. Note that due to the resource costs of the RIS 
and ARIS algorithms, the Conjecture 17.381 does not contradict Conjecture 17.41 even though both assume the same 
communication conditions. 

An informal description of the ARIS algorithm is as follows: at t = each node generates r random variables 
independently from an exponential distribution with rate parameter s^(0) for each I — 1,2, ... ,d. A local counter 
ki is set to zero and a local vector w,j is set to e^. Upon reception of any signal, if the transmitted counter kj is 
less than the local counter ki, then the signal is ignored, if kj = k t then the receiving node records the minimum 
between the received r random variables and local r random variables for each I = 1,2, ... ,d. The vector ibi 
maintains a record of the set of nodes that have a communication path with the node i for any given counter value. 
Whenever Wi(t) is updated to 1„, then the counter ki is increased by one, the consensus estimate Si(t) is updated 
as a running average of the inverse of the currently recorded mean of the minimum random variables for each 
£ = 1,2, . . . , d, Wi(t) is reset to e^, and a new set of rd random variables are locally generated. If kj > ki then the 
local counter ki is set to kj, the consensus estimate §i(t) is set to the received estimate Sj, Wi(t) is reset to and 
updated by Wj, a new set of rd random variables are locally generated, and the node i then records the minimum 
between the received r random variables and the newly generated local r random variables for each I — 1,2, ... ,d. 
It is not difficult to see that the local counter of each node will approach infinity iff Cjo,^] satisfies the IVSC 
condition (1351) . thus each element in the consensus estimate §i(t) of each node i E V will approach the inverse of 
the mean of the minimum of infinitely many random variables. By a well-known property of the minimum of a set 
of independently generated exponential random variables, together with the strong law of large numbers, we thus 
make the following conjecture. 
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Conjecture 7.39: For any positive integer r, the ARIS algorithm ()160[) — (|166[) implies |6]) at time t = ti(+) iff 
the communication sequence C[o.ti] satisfies the IVSC condition (|35j) . 

The above conjecture is significant because, besides the flooding technique and the DA algorithm (fT5j) — (fT9l) , 
there is no other consensus protocol in the literature that guarantees average-consensus for every communication 
sequence Cro^i that satisfies the IVSC condition (|35|) . 

C. Resource Cost Derivations 

In this section we explain how the entries of Table I in Sec lV-AI are obtained. We will assume an arbitrary vector 
v E K m requires 2m scalars to be defined, and similarly, an unordered set of scalars S with cardinality \S\ requires 
| S | scalars to be defined, that is 

Resource cost of v E W" = 2m, 

(167) 

Resource cost of S = \S\. 

The rationale for (|167[) is that each element in v requires one scalar to define the location of the element, and 
one scalar to define the value of the element itself. The location of each scalar in S is irrelevant because S is an 
unordered set, thus S can be defined using only \S\ scalars. An alternative to (I167[) is to assume that an arbitrary 
vector v E W n requires m scalars to be defined. Although this alternative will imply different entries for the exact 
values in Table I, the order of the storage and communication costs under each algorithm would remain the same. 
We adhere to (|167p for our resource cost computations due to its relative precision. 

Next observe that under the BM, OH, and DDA algorithms each normal consensus estimate Vi(i) will contain 
only elements belonging to the set {0, — }, similarly each vector Wi(t) under the ARIS algorithm contains only 
elements belonging to the set {0, 1}. Under the BM, OH, and DDA algorithms we can thus define each 0j(£) based 
only on the set Vi(t), 

\ {-I : ««(t)=Q} ifmSi(f)'l„>l/2, 
Vi\t) = < (168) 
[ { £ : v u {t) = i} if nvi(t)'l n < 1/2. 

A set Wiif) can be defined analogous to (I168[) to specify Wi(t). Note that both Vt(t) and Wi(t) are unordered sets 
of scalars, and thus, assuming average-consensus has not been obtained, we have from (|168[) . (|45]) . and (|162[) . 

1 < {\Vi(t)\,\Wi(t)\} < [n/2\ . (169) 

Under the condition (|45]), applying ()167D and ()169|) to the algorithm update equations (flO]). (fTT) . pO]). (|2T]) . (1271) , 
(|281) , then yield the respective upper and lower resource costs for the BM, OH, and DDA algorithms stated in Table 
I in Sec lV-AI Likewise, applying (|167p and (1169[) to (|160l) - (|165[) yield the respective upper and lower resource 
costs for the ARIS algorithm. As detailed in (|156[) — ()158|) . the Gossip algorithm proposed in [ 3 1 requires only the 
local consensus estimate §i(t) to be communicated and stored at each node, thus the communication and storage 
cost are both fixed at 2d under this algorithm. 

We next observe that under the DA algorithm (|15[) — (|19p. if a normal consensus estimate Vi(t) contains any 
elements equal to zero then these elements may be omitted from the signal and knowledge set. Given the condition 
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(|45t it follows from (|167p that the minimum number of scalars required to define Vi(t) under the DA algorithm is 
2, while the maximum number of scalars required to define i>i(t) is In. Together with these upper and lower limits 
on the resource cost of Vi(t), applying (|167p to (JT5J) and (fT6|) then yields the upper and lower resource costs for 
the DA algorithm stated in Table I in Sec lV-AI 

D. Example SVSC Sequence 

The sequence defined in (|170p below is an SVSC sequence that implies the DA, DDA, and BM algorithm all 
obtain average-consensus at the same instant. 



It is a "unit-delay" sequence because each signal sent at time t is received at time t + 1, and if a signal is received 
at time t + 1 then the next signal is sent at time t + 2. Together with Theorem 14.31 the example (1170|) implies 
that the DA and DDA algorithms possess the weakest possible necessary conditions for average-consensus that any 
algorithm can have. In contrast, the OH algorithm does not achieve average-consensus under (|170[) . 
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